Proposed new model of sound output

[creaothceann]

Frame advance mute

I'd guess sound is muted when frames are manually advanced.

fake mute desync workaround, mix interval

No idea either.

anti-resonance's sampling method (if it's better, why not just use it?)

Probably from a time when the default "sampling method" wasn't accurate either.

reverse stereo (how about just hooking up the speakers correctly?), interpolation (as if anyone actually prefers this be disabled)

*shrug* They're easy to code options. More control = better, imo.

the one thing I can't get is accurate, crackle free sound.

Tried increasing the buffer size?

[burning shadow]

If you going to implement internal resampling, the best option would be to allow user to select desired sample rate and just resample everything to it.. Maybe this is exactly what you were planning, I'm not sure..

BTW, since you are going make some changes to audio configuration, maybe it's possible to add an option to select sound card to use?

[Verdauga Greeneyes]

If you going to implement internal resampling, the best option would be to allow user to select desired sample rate and just resample everything to it..

I have to agree with this, if only because going by the last few pages, determining the internal sample rate seems like an unreliable process. Perhaps have a menu similar to speed regulation - or put it in the advanced settings; perhaps have 0 mean automatic detection if you do..

[grinvader]

fake mute desync workaround

Snes9x is also having nondeterministic core desync issues. That option works around some, but is a hackish solution - hence optional.

[Arbee]

Well from my experiences with some people, they feel that thier methods are superior to others. I had run ins, with the MAME team for this reason, and this is why I personally can't stand the project.

As I recall, you chose to have that run-in by coming to our board and flaming MAMEdev. It's not like MAMEdev had tried to stop people from using whatever the hell it is emulator you work on, and we certainly aren't stopping you from making or releasing it.

It's a difference in philosophy. MAME and bsnes exist to emulate the actual hardware behavior of the machine as accurately as is possible. In other words, we get our jollies from outsmarting Japanese electrical engineers. You're into emulation because you like to play the games. That's not an invalid position, and frankly it makes life much simpler. Your test for shipping is "am I enjoying playing Kirby?", not "does byuu's !@#$% scanline timing test work?". (On the plus side for accuracy guys, we have a real objective criteria to compare our emulation against, so when byuu says bsnes is better he ain't bragging because he can prove it. On the plus side for you, you don't have to bother).

(I recall having a flamewar with MooglyGuy over HLE and labelling people as "ROM kiddiez")

If you're talking to MG and some sort of argument doesn't break out, you're probably doing it wrong. He argues with everyone and everything: me, Aaron Giles, his mother, small cute kittens, major world religious leaders, the quadratic formula, it doesn't matter.

And you've gotta admit even if you're writing an inaccurate emulator for traditional hardware that HLE as done in N64 emulators is crappy. I can assure you that Rock and Roll Racing on the GBA will boot and run (a bit glitchily) with the SWI (BIOS call) opcode doing nothing, but that doesn't mean I should release an emulator that behaves that way.

You need to stop being upset about this stuff. It's just emulation, it's not like it's emacs vs. vi or something.

[mudlord]

As I recall, you chose to have that run-in by coming to our board and flaming MAMEdev. It's not like MAMEdev had tried to stop people from using whatever the hell it is emulator you work on, and we certainly aren't stopping you from making or releasing it.

I did not flame MAMEdev.

All I did was ask some simple questions on thier ethics, out of curiosity. That is all. How is that flaming?

You're into emulation because you like to play the games.

Stop making bold accusations as to why I do emulation. It personally gets way under my skin that the MAME team sees me as that.

Your test for shipping is "am I enjoying playing Kirby?", not "does byuu's !@#$% scanline timing test work?"

Again, cut the crap. Really.

And you've gotta admit even if you're writing an inaccurate emulator for traditional hardware that HLE as done in N64 emulators is crappy.

Yes, true.

You need to stop being upset about this stuff. It's just emulation, it's not like it's emacs vs. vi or something.

Well, I would stop it if I knew MAMEdev had a open mind towards emulation methodologies that aren't thier own. But with my run in with MG, I honestly don't know what to think.

Bye.

[tetsuo55]

And yet then again, we're around the Nyquist limit for audio already, so no human will probably be able to perceive the difference. Though that won't stop lots of people from saying they do, of course. Audio is well known for placebo effects.

I just read up on the Nyquist limit..

I'm not 100% sure what you mean so...

Sample rate is not the same as the hearable frequency.

These should be facts:

The samples played can have any frequency that fits within the bitrate.
The sample-rate is how often the frequencies are stored.
The Nyquist limit states that only half of the the sample rate is represented accurately.

So with the above we know that the 32040 out of the snes represents 16020 of accurate audio datata and 16020 of innacurate audio data.

Double the accurate audio data is enough to represent it, so put simply any value equal or higher than 32040 should represent the data accurately.

So a standard rate like 48khz is more than enough to represent 32040 accurately, as long as you do a simple resample without changing pitch.

if you want to take it a step further and accurately represent 32040 go for 96khz

penguin edit: dude, thousand and forty is 1040, not 10040.

[blargg]

And yet then again, we're around the Nyquist limit for audio already, so no human will probably be able to perceive the difference. Though that won't stop lots of people from saying they do, of course. Audio is well known for placebo effects.

I just read up on the Nyquist limit.. I'm not 100% sure what you mean so... Sample rate is not the same as the hearable frequency.

Correct. The Nyquist limit is simply the maximum frequency representable in the sampled signal (half the sampling rate). For audio, the sample rate is typically not much more than twice the limit of human hearing, because much more leads to waste (audiophiles' auditory hallucinations aside).

[tetsuo55]

And yet then again, we're around the Nyquist limit for audio already, so no human will probably be able to perceive the difference. Though that won't stop lots of people from saying they do, of course. Audio is well known for placebo effects.

I just read up on the Nyquist limit.. I'm not 100% sure what you mean so... Sample rate is not the same as the hearable frequency.

Correct. The Nyquist limit is simply the maximum frequency representable in the sampled signal (half the sampling rate). For audio, the sample rate is typically not much more than twice the limit of human hearing, because much more leads to waste (audiophiles' auditory hallucinations aside).

Well not totally wasted, looking at the most extreme individual, he/she might be able to hear 10-23000 instead of the normal max of 15-22000.

Now what about the other frequencies, now 10 to -?? can be felt as well as some of the frequencies above 23000.

So higher frequency does have some effect, you would need something that actually creates those kinds of frequencies, record it with something that is able to record at 48khz, something that is able to store it and play it back, but most importantly: speakers able to represent the sound.

Basically everything above 48khz is a waste of bandwith unless you have the equipment to accurately represent the feelable frequencies. Now they can add a lot to the overal experience, more like being there while the audio was being recorded, but the effect is minor at best and probably 99% of end users won't notice the difference anyway.

The only thing that higher sampling rates does represent the lower hearable audio better than the lower sampling rate, but as the niquist limit states, those improvements are almost negligable.

[augnober]

When people talk about the limits of human perception and a certain level of technology that is sufficient in producing anything we can perceive, I'm always a bit skeptical. Bad information gets passed around based on misinterpretation of experimental results. Sometimes people stubbornly hold that 60Hz is more than sufficient for displaying all that a human can see, even for cases where it is contradicted by what you can see. The idea may have some basis in truth, but there is a big caveat. Our senses are stimulated at all times (within reason), rather than just at discrete snapshots of time. For this reason, our eyes receive a good sweep when we see motion in the real world, while when we receive images from a display, we're only getting discrete frames. For this reason, if you want to display authentic motion at a low framerate, it is essential to implement good motion blur. This is demonstrably true since a 24Hz movie from film can look far more continuous than a 60Hz game lacking blur. And even with the motion blur, it isn't certain that a blurred sweep is going to affect us exactly as a directed sweep would, because perhaps there are aspects of our ability to sense motion that we still don't understand. So I'd be careful to make sure that something similar doesn't need to be taken into account for audio.. Though I'm not meaning to say this stuff applies to SNES emulation -- since the goal is basically to reproduce the SNES, warts and all . I was glad to see it pointed out that the Nyquist frequency is a technical/mathematical/engineering phenomenon, rather than something about us or nature. It's the greatest possible frequency of change (ultimately toggling) for a certain sample rate.. since at some point, if the toggling were any quicker, the toggling would sometimes occur twice before the next sample is taken (which negates itself and is not a toggle at all).

[Deathlike2]

Considering all the audio problems others have in general with this emu, people complaining about byuu wanting to make it nicer for their sound card (particularly with bad drivers and/or for Linux audio in general) are just stupid for not understanding the benefits given the state of affairs.

It is not as if critical stuff is being altered to change emulation or anything of the sort.

[FitzRoy]

Considering all the audio problems others have in general with this emu, people complaining about byuu wanting to make it nicer for their sound card (particularly with bad drivers and/or for Linux audio in general) are just stupid for not understanding the benefits given the state of affairs.

It is not as if critical stuff is being altered to change emulation or anything of the sort.

First, I don't think audio problems are all that common with bsnes. I certainly haven't seen many posts to believe that.

Second, the argument could be made that when you prop something like that up, you inadvertently get more of it. Because it prevents the true consequence of such drivers, either complaints from consumers or lost sales to a better product. And that eliminates the incentive for these issues to get fixed on the right end.

[Deathlike2]

Considering all the audio problems others have in general with this emu, people complaining about byuu wanting to make it nicer for their sound card (particularly with bad drivers and/or for Linux audio in general) are just stupid for not understanding the benefits given the state of affairs.

It is not as if critical stuff is being altered to change emulation or anything of the sort.

First, I don't think audio problems are all that common with bsnes. I certainly haven't seen many posts to believe that.

When Triple Buffering was implemented way back when in BSNES, there were plenty of audio issues that came with that option. The way BSNES renders video (not using VSync that's built into DX itself) is half the problem.. which in turn hurts audio playback. This is a self-inflicted issue and frankly if byuu doesn't want to use that, then he better use the solution that he suggest. The number of users that have been mentioning audio problems for BSNES (however slight, and not even factoring the Linux port) far outnumbers the same kind of problems ZSNES has ever gotten for its Windows port, let alone Snes9x.

Second, the argument could be made that when you prop something like that up, you inadvertently get more of it. Because it prevents the true consequence of such drivers, either complaints from consumers or lost sales to a better product. And that eliminates the incentive for these issues to get fixed on the right end.

I will agree that this is a inherant problem with drivers these days, but some of it is a hardware issue (and for some, people's paranoid to upsampling in DX). However, if one truly wants people to experience SNES audio the way it was originally played back (assuming you are rendering at full fps), it would benefit BSNES more than any other emu as of this moment. This in turn can help the Linux port as well. Native hardware formats nowdays won't include 32000Hz to begin with, and that unfortunately won't change because the SNES community demands for it.

[Arbee]

On topic content: all SB Live and Audigy cards have a native rate of 48000 Hz, so if you feed them anything else they run it through an 8-point spline resampler[1]. I run everything possible at 48 kHz for that reason (X-Fi cards don't work well on Linux yet and I don't know what's going on there).

[1] For Audigy 2s at least, it can actually output up to 192 kHz, but anything below 48 is resampled to 48 as far as I know.

Now, a quick trip back to that other stuff.

Stop making bold accusations as to why I do emulation. It personally gets way under my skin that the MAME team sees me as that.

Ok. So why are you into emulation if it's not to play games and it's not to find out how hardware works?

Well, I would stop it if I knew MAMEdev had a open mind towards emulation methodologies that aren't thier own.

Three things here:

1) Why care what MAMEdev or any of it's members thinks? Seriously. You may as well develop an ulcer from byuu's theory of licenses.
2) MAMEdevs on random message boards are not speaking for MAMEdev. Not Aaron Giles, not me, and definitely not MG. Official team opinions are on mamedev.org, period, and I'm pretty sure there's no statement on this topic there at all.
3) 4 different MAMEdevs have released and maintain inaccurate fast emulators. As byuu has noted, all the really nasty commentary comes from people with a sweet Gateway their mom just bought them and they get really angry when bsnes or MAME doesn't run fast enough on it. So the closest you'll see to an official statement on this is that we encourage people to write faster-than-MAME emulators, never mind how. It keeps all those poor sweet Gateways busy.

[byuu]

Sample rate is not the same as the hearable frequency.

I was more referring to that humans can usually hear between ~1500hz and ~20000hz. If you cut the SNES frequency in half for Nyquist, you're still pretty close to the upper limit. But meh, nevermind, not important really.

The way BSNES renders video (not using VSync that's built into DX itself) is half the problem.. which in turn hurts audio playback. This is a self-inflicted issue and frankly if byuu doesn't want to use that, then he better use the solution that he suggest.

If you're going to comment on this, the least you could do is read my past discussions on the matter. I've already tried to use vsync. I can do that just fine, but then I get differing numbers of samples each frame. The difference is so severe that resampling each batch results in audible pitch differences. If I buffer out the differences, that just results in bigger pitch changes, just less frequently.

I admit, I'm not smart enough to figure it out. I know most other emulator authors have. If you're that much of an expert, please do consider helping. I believe many of us would be willing to pay you for your time, as well.

The number of users that have been mentioning audio problems for BSNES (however slight, and not even factoring the Linux port) far outnumbers the same kind of problems ZSNES has ever gotten for its Windows port, let alone Snes9x.

That's not at all surprising. You guys focus on the end user experience, I focus on the core. And the results of all emulators reflect that. If I had been working on bsnes since ~1997, and had 5-10 developers, I'd probably have great audio hardware support, too.

However, if one truly wants people to experience SNES audio the way it was originally played back (assuming you are rendering at full fps), it would benefit BSNES more than any other emu as of this moment.

And now you're exaggerating. As Arbee was saying, the sound card will just use an 8-tap spline filter to convert 32khz to native output. I've tried with anything from point filters (worst quality) to hermite filters (best quality), and couldn't perceive any difference in upscaling to the native soundcard rate. I don't think this change will even make a difference.

Right now, bsnes is the only emulator to sync purely to audio output, with no internal resampling to allow syncing to the video refresh rate properly, so it's still going to have the best sound around. And the worst video, bar none.

You may as well develop an ulcer from byuu's theory of licenses.

:P

[Deathlike2]

The way BSNES renders video (not using VSync that's built into DX itself) is half the problem.. which in turn hurts audio playback. This is a self-inflicted issue and frankly if byuu doesn't want to use that, then he better use the solution that he suggest.

If you're going to comment on this, the least you could do is read my past discussions on the matter. I've already tried to use vsync. I can do that just fine, but then I get differing numbers of samples each frame. The difference is so severe that resampling each batch results in audible pitch differences. If I buffer out the differences, that just results in bigger pitch changes, just less frequently.

I admit, I'm not smart enough to figure it out. I know most other emulator authors have. If you're that much of an expert, please do consider helping. I believe many of us would be willing to pay you for your time, as well.

I'm not that bright either. I have not written any sound code, and frankly I still have to figure it out some time. Heck, I claim to know nothing, or a little of something (other things unrelated to emulation in general).

However, when many other devs have repeated the same answer to this issue and you still argue that "it doesn't work", obviously something is wrong.

The number of users that have been mentioning audio problems for BSNES (however slight, and not even factoring the Linux port) far outnumbers the same kind of problems ZSNES has ever gotten for its Windows port, let alone Snes9x.

That's not at all surprising. You guys focus on the end user experience, I focus on the core. And the results of all emulators reflect that. If I had been working on bsnes since ~1997, and had 5-10 developers, I'd probably have great audio hardware support, too.

Do I have to dig up responses that other devs have said towards this matter? (Though, I get the feeling tons of that have died in the huge thread o' doom). It doesn't come from me. It comes from the same people who have regularly contribute. I'm not saying everything said applies, but on the other hand, it is not as if they had similar trouble AFAIK, and not being an SNES specific detail (in some instances, it is, but it can't be every instance).

You may as well develop an ulcer from byuu's theory of licenses.

The fact that even that licenses ever gets discussed is mindblowing, and not sane either.

[Verdauga Greeneyes]

not using VSync that's built into DX itself

That VSync is unreliable as hell, even if you use D3DPRESENT_INTERVAL_ONE rather than D3DPRESENT_INTERVAL_DEFAULT. (the former uses timeBeginPeriod(1) to set system timer resolution from the default 10ms to 1ms) Trust me, I've experimented with both. The only way to get VSync to work properly (that is, consistently, and without taking up 100% CPU) is to create a new thread that generates a VSync event that another thread in an alertable wait state can respond to. This thread sleeps for about one millisecond less than the time between VBlanks (to accomplish just that much you also need timeBeginPeriod(1) ), then polls GetRasterStatus() until it's in VBlank. Problem with that is that it probably works best on a multi-core system. Thankfully, there probably aren't any single core systems that can run bsnes at full speed anyway

Speaking of which, I should really finish the library I was writing to accomplish all that without too much of a hassle. I got a bit too ambitious - the monitor -can- generate VBlank IRQs, and they are handled by the Video Miniport driver, so maybe if I could slide a driver of my own in underneath the GPU manufacturer's Miniport driver, I could have it signal an Event on VBlank.. Anyway, I'll hopefully turn my attention back to that library-to-be in the coming weeks.

[augnober]

The only way to get VSync to work properly (that is, consistently, and without taking up 100% CPU) is to create a new thread that generates a VSync event that another thread in an alertable wait state can respond to. This thread sleeps for about one millisecond less than the time between VBlanks (to accomplish just that much you also need timeBeginPeriod(1) ), then polls GetRasterStatus() until it's in VBlank. Problem with that is that it probably works best on a multi-core system.

I used the same method (a timer that expires just before refresh, and then polling) before when I was making hacks to Dosbox. It worked decently. I didn't bother to put it in another thread though.. and I didn't think of creating a vsync event. I've got to try that sometime. If you're syncing a pageflip to vsync, some drivers (my up-to-date nvidia driver on Windows for example) are nice enough to do it cooperatively instead of busy-waiting, so that could simplify matters depending on what you're doing. You probably don't want your program's performance to be too much at the mercy of the video driver in that respect though, because drivers have gotten it wrong in the past.

If you write any vsync-related code that runs correctly on Linux and/or multiplatform.. or that generally helps manage those issues, I'd be interested. I've always been able to solve my vsync issues to my satisfaction on Windows, but never on Linux (I usually don't look much beyond sdl, which has had unsatisfactory/unreliable refresh capabilities in the past as far as I could tell).

[Verdauga Greeneyes]

If you write any vsync-related code that runs correctly on Linux and/or multiplatform.. or that generally helps manage those issues, I'd be interested. I've always been able to solve my vsync issues to my satisfaction on Windows, but I've never been convinced that I've handled it reliably on Linux.

I'm not yet at that stage, but a universal solution would be my ultimate goal. At the moment I have no experience with Linux-related APIs, so it'll probably be a while.

User Mode APCs and alertable wait states seem to work beautifully to solve this problem (of course the main annoyance is that there's no avoiding at least -some- polling, when the OS should just be generating an event for this in the first place), but I'm a bit worried about -their- latency. In my tests, I didn't get any frame drops on a multicore system at 75Hz, so presumably it works around the 1ms res. system timer somehow (VBlank only lasts for maybe half a millisecond), but I'd still like to do some testing on this..

In my somewhat complicated setup, I had:
A thread that waits until VBlank, then sends out a QueueUserAPC for a Present operation
A thread that waits until a frame is ready, then sends out a QueueUserAPC for a backbuffer update
A thread that handles the queue, ensuring that no Present operation would occur during a backbuffer update
A thread that draws an example animation then signals an event (that auto-resets when handled) at a user-defined speed, and uses a 3-buffer system to avoid locking.

This worked quite well, so inter-thread communication seems to be pretty rapid.

[byuu]

However, when many other devs have repeated the same answer to this issue and you still argue that "it doesn't work", obviously something is wrong ... Do I have to dig up responses that other devs have said towards this matter?

Frankly, I don't understand why you keep bringing this up at all. You aren't contributing anything useful to this matter. I never argued that what others were saying was wrong. I specifically said that I cannot figure out how to implement it in bsnes.

Meaning, yes, for the tenth time, I'm not a good sound programmer.

I've tried for three years now and failed every time. Clearly I'm not able to comprehend the process. Nobody else during that time has submitted actual code (nor should they have to), just explanations that don't help me. Thus, it's not going to get done.

Are you happy now? Or do you want to keep bringing up how I'm obviously not capable of doing this, as if it's somehow news to anyone here?

[Deathlike2]

I'm sorry. I leave you to your own devices regarding this matter. I did not mean to imply any sort of imcompetance on your end.

[byuu]

I did not mean to imply any sort of imcompetance on your end.

Your previous posts most certainly did, however.

"self-inflicted issue and frankly if byuu doesn't want to use", "The number of users that have been mentioning audio problems for BSNES ... far outnumbers the same kind of problems ZSNES has ever gotten for its Windows port, let alone Snes9x", "you still argue that "it doesn't work", obviously something is wrong", "Do I have to dig up responses that other devs have said towards this matter?"

That all comes across as very confrontational to me.

If that truly wasn't your intention, then I apologize.

But it is a fair criticism, everything you said is true. I am incompetent at sound design. It's just not productive to keep bringing it up, when I've already admitted it.

But then, I do the same thing about core emulation issues to you guys, so par for the course, right?

[Deathlike2]

Many people have complained about audio and ALSA on the boards. Heck, we even have a thread with people bitching and whining about it too. We're just as culpable.

I didn't mean to take this seriously as it did. Perhaps, it is simply not a good day for me, but it is my fault regardless. My apologies.

[byuu]

Many people have complained about audio and ALSA on the boards. Heck, we even have a thread with people bitching and whining about it too. We're just as culpable.

It really isn't ZSNES' fault that ALSA output sucks, though. ALSA is just a terrible API. I can get perfect sound out of WaveOut, DirectSound, libao, OpenAL and OSS; and ZSNES can even get somewhat decent sound out of SDL, of all libraries. That neither of us can get ALSA working really says more about its API than about us.

RedDwarf's recent ALSA stuff is very promising though.

[grinvader]

I was more referring to that humans can usually hear between ~1500hz and ~20000hz.

Baw ? What about the very hearable 20-1500 range ?
I've even met some people who could even hear stuff down to 12Hz (bad joke in electronic class == find out who is superman !).

On the other side, you can safely assume anyone above 40 can't hear above 16-18k unless he lived a pristine life devoid of any major noise.