bsnes vsync development thread

[byuu]

Are there TFTs that support that? :?

I wish more would support 50hz. At least without requiring custom Linux modelines or crazy Windows shareware. That really doesn't seem like it'd be very hard for these monitors to do, and it definitely has its use.

[Verdauga Greeneyes]

I think my laptop goes to 50Hz sometimes to save power. Annoyingly it does this while reporting 60Hz - although I think power saving is broken entirely in Vista x64 (due to some odd BIOS bugs)

[blargg]

While explaining the subtleties of linear interpolation of an unending sample stream to byuu, his preference for resampling immediately rather than buffering it as I usually do prompted this absolutely simple and correct implementation of linear interpolation. It would of course use fixed-point (all-integer) arithmetic in practice.

Code: Select all

double step = 32040.0 / 48000.0;

void play_sample( int );

// SPC-700 DSP emulator calls this each time it generates a sample
void dsp_sample( int curr )
{
    static int    prev = 0;
    static double frac = 0.0;
    
    // Interpolate and play whatever samples can be generated from
    // prev and curr
    do
    {
        // Interpolate between prev and curr sample, based on
        // frac, where 0.0 is entirely prev sample, and 1.0 is
        // entirely curr sample.
        double left = 1.0 - frac;
        int interp = (prev*left + curr*frac) / 2;
        play_sample( interp );
        
        frac += step;
    }
    while ( frac <= 1.0 );
 
    prev = curr;
    frac -= 1.0;
}

[Verdauga Greeneyes]

So in integer terms that would be

Code: Select all

int in_rate = 32040;
int out_rate = 48000;

void play_sample( int );

// SPC-700 DSP emulator calls this each time it generates a sample
void dsp_sample( int curr )
{
    static int prev = 0;
    static int frac = 0;
   
    // Interpolate and play whatever samples can be generated from
    // prev and curr
    do
    {
        // Interpolate between prev and curr sample, based on
        // frac, where 0 is entirely prev sample, and out_rate is
        // entirely curr sample.
        int left = out_rate - frac;
        int interp = (prev*left + curr*frac) / (2*out_rate);
        play_sample( interp );
       
        frac += in_rate;
    }
    while ( frac <= out_rate );
 
    prev = curr;
    frac -= out_rate;
}

? (I was bored)

[blargg]

I suppose that would work, but it's unconventional and inefficient since it involves a divide. Generally the fixed-point unit is a power of 2, allowing a right shift to convert to integer. To convert to fixed-point, shift floating-point values left by the number of fraction bits (that is, multiply by the equivalent). To convert back to integer, shift right the same. To simplify code, I make the unit constant equal to 1.0 in fixed-point.

Code: Select all

typedef int fixed;
int const bits = 15; // number of bits in fraction
fixed const unit = 1 << bits; // 1.0 in fixed-point

fixed step = 32040.0 / 48000.0 * unit + 0.5; // +0.5 to round to nearest

void play_sample( int );

void dsp_sample( int curr )
{
    static int   prev = 0;
    static fixed frac = 0;
    
    do
    {
        fixed left = unit - frac;
        fixed interp = (prev*left + curr*frac) / 2;
        frac += step;
    
        play_sample( interp >> bits );
    }
    while ( frac <= unit );
    
    prev = curr;
    frac -= unit;
}

[FitzRoy]

When this is working and no longer needs hidden, can I suggest some changes to the settings menu to accommodate it?

Code: Select all

Video Mode >
    Windowed
    Fullscreen
    -----
    Sync to Audio

Video Resolution >
    NTSC
    PAL
    -----
    Correct Aspect Ratio
    -----
    Scale 1x
    Scale 2x
    Scale 3x
    Scale 4x
    Scale 5x

Video Filter > (unchanged)

Video Frameskip > (removed)

Order in resolution seems more logical this way. Definition of the resolution should come before actions applied to it. And I thought it would be nice to make the modes visible and selectable with the mouse.

And as always, the menubars should be locked to a toggle and taken out of the menu, and mute audio output should be changed to "(X) Audio Output" or "Audio Output > () Mute".

And thanks to fitzroy for making his views clear in his full reply. I of course understand the frustration in having the windows port held back

I'm not actually sure if linux is the problem this time, I can't understand any of their attempts to put the problem into layman's terms. I just remember when vsync worked for me in the windows-only days and am stirring the pot as usual. Changes since then with libco and everything have probably complicated matters. Meh, who cares, it's getting fixed and it's going to be awesome.

[byuu]

While I strongly agree with your logic for the resolution menu ordering, the reason I keep it the other way is because Scale nX is changed so much more frequently than NTSC / PAL. I imagine most people will leave that setting alone even when switching games. In fact, I've wanted to add an auto-detect setting to that which switches modes on game load if needed, I just haven't had the time and desire to do that yet.

And as always, the menubars should be locked to a toggle and taken out of the menu, and mute audio output should be changed to "(X) Audio Output" or "Audio Output > () Mute".

Take menubars out of the menu? And for audio, I might do Audio -> "100%, 75%, 50%, 25%, Mute" ... I don't want to scale the audio to be louder, to avoid aliasing or whatever else is possible when you compress the range to be louder.

I'm not actually sure if linux is the problem this time, I can't understand any of their attempts to put the problem into layman's terms.

I lost triple buffering with the true fullscreen mode, but it's not so bad. The video tends to tear a bit if you fill 100% of the screen at a high resolution now (1920x1200 on mine), but even with a little border it's fine. But the benefits to not using triple buffering are worth that con:

- You can switch between windowed and fullscreen immediately; whereas some monitors can take several seconds to truly change resolutions.
- You don't have to have controls for resolution and refresh rate, that may possibly be invalid anyway.
- It doesn't screw up and shrink all your other open windows.
- You can quickly and easily alt-tab to other applications if needed.
- You can actually see the menubar and GUI stuff without having to return to windowed mode (note that D3D's option to allow that + page flipping was really just internally using vsync)
- I don't have to worry about how the hell you change resolutions in Xorg :P

I'm really sorry that the Linux port has and continues to hold back the Windows port while I catch up to speed. But do keep in mind my primary OS is Linux. To hell with a migration path to DRM Vista, may as well get used to Linux now. I'm not about to be the guy running XP in 2016 and complaining when other apps finally stop supporting it, either :P

[FitzRoy]

While I strongly agree with your logic for the resolution menu ordering, the reason I keep it the other way is because Scale nX is changed so much more frequently than NTSC / PAL. I imagine most people will leave that setting alone even when switching games. In fact, I've wanted to add an auto-detect setting to that which switches modes on game load if needed, I just haven't had the time and desire to do that yet.

Hmm, an inch lower for twice the sense making? I'll take it. And I don't even touch my video settings after I set them, who are these people resizing every session?

Take menubars out of the menu?

You remember how someone was hiding the statusbar via the menu, then using the menubar toggle, hiding the menubar while inadvertently showing the statusbar because the the statusbar defaults to the same hotkey? At which point, they toggled the menubar in order to change the toggle or re-enable the statusbar so that the toggle would work on both.

This only happened because of that menu option. The answer is not to keep the menu option and leave the statusbar unassigned, forcing the majority to go in and assign it. The answer is to remove the menu option. It adds nothing over the hotkey because it can't be toggled when the menu is invisible. And it causes the above because it exists in addition to the hotkey. The toggles for both bars should be hotkeys only, and default to the same key. That way, the minority who want to hide one and not the other won't confuse themselves.

And for audio, I might do Audio -> "100%, 75%, 50%, 25%, Mute" ... I don't want to scale the audio to be louder, to avoid aliasing or whatever else is possible when you compress the range to be louder.

Well what's the benefit of that? I thought we agreed that volume was slider or bust. It won't look weird in the config to have an Audio section with a single volume slider. The only thing that looks weird in a fixed size config is bloat added to fill it up. The space is there to accommodate sections that need it, not to be filled by people who only like trees during christmas. Here's what you do: add an Audio section below video and put that slider in there. In the menu, change it to "(X) Audio Output." It's not a muter, it's an enabler, while volume is defined in the config. If that checkmark is not there, it doesn't set the slider to 0, it just "disables" it.

Lastly, can you put "Input" first or remove that automatic thing? I only suggested it on the condition that it be put first, it makes no sense to arrive at the middle of a list every time.

[byuu]

Okay, here is my resampler:

Code: Select all

  void resample(uint16_t l, uint16_t r) {
    static int pl = 0, pr = 0;
    static int frac = 0, step = 32000.0 / 32100.0 * 32768.0 + 0.5;

    int cl = (int16_t)l, cr = (int16_t)r;

    while(frac <= 32768) {
      int sl = (pl * (32768 - frac) + cl * frac) >> (15 + 1);
      int sr = (pr * (32768 - frac) + cr * frac) >> (15 + 1);
      sample(sl, sr);

      frac += step;
    }

    pl = cl, pr = cr;
    frac -= 32768;
  }

Works fine, I can output audio at 44100hz, and skew the input at 32040hz, and audio sounds great. Can't tell the difference between that and normal 32000hz. Took me a while to remember that the S-DSP outputs signed 16-bit audio, though, heh. Sounded quite bad before I sign-extended the samples.

But let's ignore that for a moment and look at the actual sync code. Okay, in void Video::update(), it simply calls D3D->Present(), which will wait for vsync and then blit to the screen.

I added time_t n = clock() before and after the call, and printed the results. At 60hz refresh with a NTSC game, I get:

Code: Select all

* video update = 16ms
* video update = 16ms
* video update = 15ms
* video update = 16ms
* video update = 16ms
* video update = 15ms
* video update =  0ms
* video update =  0ms
* video update =  0ms
* video update =  0ms
* video update =  0ms
* video update =  0ms
* video update =  0ms
* video update =  0ms
* video update = 15ms
* video update = 16ms
* video update = 16ms
* video update = 15ms
* video update = 16ms
* video update = 16ms
* video update = 15ms
* video update =  0ms
* video update =  0ms

I get the exact same pattern with or without audio also waiting for the playback cursor to catch up. It's never a constant rate, and I have absolutely no idea why not.

I even tried a manual "poll for vblank" loop, with D3D's wait for vblank to blit turned off internally, forcing 99% CPU usage in the process, and I still got the same pattern.

But, at least it's never >~16ms, right? So, in theory, so long as our audio buffer always has more than ~16ms of samples, we should be fine. We probably want up to ~24m-32s or so just to be safe, for emulation overhead and stuff.

So, let's ignore resample() for now. I instead overclocked the S-DSP to 32100hz sample generation.

Now, let's say I turn off vsync entirely, let emulation run as fast as possible, just to see what kind of steady rate of input to output samples I generate:

Code: Select all

  void sample(uint16_t, uint16_t) {
    if(++data.buffer_pos & 15) return;

    DWORD ring_pos, pos, size;
    dsb_b->GetCurrentPosition(&pos, 0);
    ring_pos = pos / data.ring_size;
    if(data.ring_pos != ring_pos) {
      printf("* %5d input to %5d output\n", data.buffer_pos, data.ring_size / 4);
      data.ring_pos = ring_pos;
      data.buffer_pos = 0;
    }
  }

*  2640 input to  1600 output
*  2656 input to  1600 output
*  2592 input to  1600 output
*  2640 input to  1600 output
*  2640 input to  1600 output
*  2656 input to  1600 output
*  2640 input to  1600 output
*  2656 input to  1600 output
*  2656 input to  1600 output
*  2656 input to  1600 output
*  2656 input to  1600 output

I rounded to 16, because GetCurrentPosition() takes over 1ms per call to complete. As you can see, for each emulated frame, I'm getting a very constant stream of input samples from the SNES. I was running at ~150% SNES speed as you can see, because neither video nor audio were blocking, hence I had uncapped speed.

Okay, so now let's allow both video to block with vsync, and audio to block with waiting for the playback cursor:

Code: Select all

  #define RINGS      6
  #define LATENCY 1600

  void sample(uint16_t l_sample, uint16_t r_sample) {
    //static int spos = 0;
    //if(++spos >= 360) spos = 0;
    //l_sample = r_sample = sin(spos * M_PI / 180) * 0x2000;

    data.buffer[data.buffer_pos++] = (l_sample << 0) + (r_sample << 16);
    if(data.buffer_pos < LATENCY) return;

    static int rdring = 0, wrring = RINGS - 1;

    DWORD pos, size;
    time_t start = clock();

    while(true) {
      dsb_b->GetCurrentPosition(&pos, 0);
      rdring = pos / data.ring_size;
      int distance = (RINGS + wrring - rdring) % RINGS;

      if(distance >= 2 && distance <= 4) break;
      //Sleep(1);
    }

    void *output;
    if(dsb_b->Lock(wrring * data.ring_size, data.ring_size, &output, &size, 0, 0, 0) == DS_OK) {
      memcpy(output, data.buffer, data.ring_size);
      dsb_b->Unlock(output, size, 0, 0);
    }

    wrring = (wrring + 1) % RINGS;

    time_t end = clock();

    if(end - start > 0) printf("* audio update = %2dms\n", int(end - start));

    data.buffer_pos = 0;
  }

A walk through the code ...

I have six buffers, and each are 50ms in length (32000hz/1600 samples = 50ms.)

In theory, this means that at any given point, audio should be able to run by itself for at least 50-100ms before running out of samples. So even if vsync blocks the system for 16ms, there will still be at least 32ms of audio available.

Now ... at the top is a simple sine wave generator, makes it easier for me to detect crackling in the audio (thanks to blargg for that idea.) It's disabled, but it really makes no difference in our case.

Next up, we write samples from emulation until we have filled our buffer.

At this point, we want to see how far apart the read and write block counters are. The idea is that we don't want to write to sample data that is currently being played.

What I do is make sure we are 2-4 blocks ahead of the playback cursor. Let's pretend we have a new emulated block, but no blocks have fully played on the sound card. Then the distance will grow longer. We're getting more input than output. My loop will block until the audio has played through a few blocks to catch up.

Now let's say we ended up playing more than one block of audio on the sound card in the time it took us to generate one emulated block. This is actually a huge problem, it means emulation is not maintaining full speed, and if we don't correct it quickly by generating more than enough audio, the audio will get bad no matter what we do.

This is why I set the S-DSP clock to 32100*768, so that it generates way more samples than it really should. blargg calculated that we'd need at least ~32048 samples / second for 32khz output with a monitor running at ~60hz to avoid generating fewer samples in emulation than are played back by hardware.

So I recorded the millisecond delay of how long my audio loop waits, if it ends up waiting at all, and I get this:

Code: Select all

* audio update = 16ms
* audio update = 16ms
* audio update = 16ms
* audio update = 156ms
* audio update = 140ms
* audio update = 15ms

That was over ~1 minute of play time, you'll see that the audio buffer underflowed twice (too few input samples to output samples.) Each time that happens, audio gets really bad for ~150ms.

[byuu]

For Windows / Direct3D / DirectSound only.

Code: Select all

http://byuu.cinnamonpirate.com/temp/bsnes_v034_wip03v.zip

Leave it at 100% speed, play NTSC games, leave frameskip off. I don't care if any of that is broken or not right now.

There are two special variables this time: system.vsync_magic and system.latency_magic.

The former is the skew for the resampler, you create that many samples per 32000 samples of output. The latter is the latency in samples. It will tell you how much total latency you'll end up getting when you start the emulator.

Note that the system requirements are much greater with the CPU<>SMP desync trick disabled. It's something like 10-20% slower. So leave off the filters, please.

If vsync_magic is too low / high, it will tell you on the terminal by printing an underflow warning. If latency_magic is too low, you'll hear crackling.

The bad news: no matter what values I plug in, I still get crackling. I can get it to be pretty rare, but I'm completely unable to get smooth audio. Maybe you'll have better luck, who knows.

For me at least, the vsync_magic value that sounds best keeps varying every few minutes between 32100 and 32250. The latency is through the fucking roof. I've got it over 120ms and it's still not enough to prevent occasional audio crackling. It's already much too high to be practical for a release.

Note that without vsync, it only needed to be 60ms, and that was a conservative number. We could get it down to 20-40ms with the right hardware.

[FitzRoy]

Yeah, I can't avoid the crackling either. ~3250 seems to give the best result here. Reducing latency doesn't seem to increate the underflows for me, it just makes the crackling last longer when they happen. It wasn't that much worse at 80ms.

[FirebrandX]

I tried out the new wip and here's what happens:

Audio plays fine for about 30 seconds, meanwhile the other window keeps updating with "1ms". Then the crackling slowly begins to start and begins to get more intense over the next 10 seconds or so. Finally the window updates with an underflow warning and the sound fixes itself. The process then repeats all over again. If I fiddle with the settings I can sometimes make it take a little longer, but the crackling eventually starts building up again.

Is it somehow possible to just play the frames in sync to the audio and then just skip one frame once the count is exactly one frame behind the audio? It seems like this is how Kega does it as far as I can tell.

[FirebrandX]

Update:

I just set the system latency to 10000 (which ended up being over 300ms) and the crackling was gone! The only thing that happens now is every 30 seconds or so there is a slight hiccup in the music, and this happens at the same time as the underflow warning.

I'm going to start lowering the latency down by increments of 1000 until I start hearing crackling again. I'll leave the audio setting at the default value for now while I do this.

[FirebrandX]

Update 2: Just tried now with 9000 latency magic (32100 set as default for vsync magic) and it took 2 minutes before an underflow finally happened! Still no crackling at all, so the sound was perfectly smooth up until the undeflow hiccup.

This is already LOADS better than screen-tearing bsnes.

[FirebrandX]

Update 3:

Ok, I've found my average cutoff for latency magic completely removing crackling. Its 5500 or 172 ms. Anything below that and I start to hear the occasional crackle, with the frequency and intensity increasing the more I lower the latency from 5500.

The underflow hiccups seem to happen completely randomly. Sometimes its a couple minutes before one happens, other times two will happen within the span of 30 seconds.

Now that I've found my target latency, I'll work on fiddling with the vsync magic setting to see if that can help on the random underflows.

[FitzRoy]

Update 2: Just tried now with 9000 latency magic (32100 set as default for vsync magic) and it took 2 minutes before an underflow finally happened! Still no crackling at all, so the sound was perfectly smooth up until the undeflow hiccup.

This is already LOADS better than screen-tearing bsnes.

Yeah, if I raise it to about 6500 (200ms), I get a slight crackle every 90 seconds or so. It's pretty good, I prefer it over tearing. But anything over 200ms results in a pretty noticeable audio delay. I'm guessing byuu can't stand anything above 120ms.

[FirebrandX]

Update 4:

I just played Super Turrican for 30 minutes straight with ZERO underflows and ZERO crackle. I finally turned the game off, having been satisfied I hit the "jackpot". The magic numbers for my system:

vsync magic: 32130
latency magic: 5500

Please note I did this test with no filters active, windowed mode, scale 3x setting.

My rig:

Maximus Formula Mobo
4 GB ram
Q6600 Quadcore
Ati Radeon 3850 HD

Desktop Monitor at 1680x1050 60Hz.


Also I'd like to add that I'm totally happy with the gameplay and audio sync at 172ms. Everything looked and sounded great as far I am concerned.

[byuu]

That's good, as long as you guys are happy, we can make it an option.

But I'm really dissatisfied with latency above 80ms, let alone 175ms ... there's no logical reason why the numbers have to be that high :/
And I also want the system requirements lowered back more. It should also be possible to set the out-of-order execution to ~2500 cycles (~3-4 samples), down from the current ~250000 cycles (~300-400 samples) and get just as good audio output as with 0 cycles in that test WIP.

I'll need that to work well, because I intend to use the same out-of-order trick for the PPU eventually (same setup, $2100-$213f = PPU black box, $2140-$217f = SMP black box)

I've been working on some Windows-only tricks to query the vblank position and wait for audio in the same loop. That basically means we can use much lower latency (as there's never a video lock eating up to ~17ms of buffer), but video is harder to get completely smooth.

But for some reason I was still getting crackling audio anyway, even though it should be impossible.

If it does work, it may be possible for Linux to do the same with glXWaitVideoSyncSGI, which is an ARB add-on. But I hear it's not really reliable.

Thus, it'd be nice to get this working without having to know the monitor's vblank status.

[FitzRoy]

Don't settle, there's no rush. Ideally, you'll get it to the point where people won't have to fiddle with multiple advanced options to get it working. Thread's still young, someone may yet spot what's amiss.

I'm really impressed with the resampling quality. Can't tell the difference, myself.

[FirebrandX]

Hey Byuu,

On a hunch, I set my latency back down to the default 3600, realizing that my zero underflow setting of 32130 may be removing the crackle as well. I was correct. Zero underflow and Zero crackle now with 32130 & 3600 respectively.

I'll keep lowering the latency to see just how well my vsync magic number works at 32130, then I'll report back here with the results.

By the way, I also did my latest test in fullscreen mode with scanlines and everything was flawless. I'm so totally stoked about this latest breakthrough in bsnes developement!

[FirebrandX]

Ok, I was able to get the latency down to 3500 and still have smooth audio. So here are my final numbers for my system:

vsync magic: 32130
latency magic: 3500

This gives me flawless audio & video for as long as I could tell while playing Super Turrican. I'm thinking of trying out other games like CT to see how they hold up with the more intensive graphics.

[King Of Chaos]

Constant crackling here regardless of settings. I'll mess with it more later and see if I can adjust it.

[FirebrandX]

Chaos, start with a really high latency number like 10,000 and check to see if the sound crackles (don't count the underflows as a crackle either). If the only hiccups you hear are the underflows, then start lowering your latency to see how low you can get it before crackling shows back up. Once you find the lowest setting with no crackle, start increasing the vsync number by increments of 10 and testing for how many underflows you get within a 3 minute period.

Really IMPORTANT note: you have to close and restart bsnes EACH time you change the magic number settings. They will NOT update on the fly.

[DancemasterGlenn]

Firebrandx: Not to be a dick, because I don't want to be, and your testing is obviously appreciated, but I would appreciate it if you used the edit button to just add to your posts if no one has posted anything in between. Thanks.

[FirebrandX]

Thanks for the advice.

Sorry that bothered you so much.