View unanswered posts | View active topics It is currently Mon Aug 19, 2019 2:08 pm



This topic is locked, you cannot edit posts or make further replies.  [ 283 posts ]  Go to page 1, 2, 3, 4, 5 ... 12  Next
bsnes v0.036 released 
Author Message
Post bsnes v0.036 released
byuu.org wrote:
2008-09-14 - bsnes v0.036 released

This release fixes a somewhat serious bug introduced in v035, and also vastly improves Windows support for non-ANSI filenames.

The bug was triggered when HDMA would occur during DMA. If the DMA were long enough, subsequent HDMA transfers would be blocked. This caused graphical glitches in Star Ocean, Super Mario Kart, and possible more games. If you noticed any regressions from v034 to v035, this was almost certainly the cause. Once again, we're operating under the assumption that there are no known bugs currently, so please let us know here if you find any.

I've also rewritten the file handling for the emulator. On Windows, attempting to load a file with non-ANSI characters (eg Russian, Japanese, etc) would cause these characters to be removed. This meant that no version of bsnes thus far could load these files. This problem was exacerbated when I ported the user interface to Unicode (UTF-16), this caused even config and locale file loading to crash the emulator.

The root of the problem is that Windows only accepts non-ANSI strings in UTF-16 format, whereas bsnes' UI wrapper converts strings to UTF-8 interally. When passing these file names to the standard file functions (fopen(), std::ifstream, etc), file loading would fail. To fix this, I replaced all file access functions with a new version that would convert the UTF-8 filenames back to UTF-16, and use appropriate access functions (_wfopen(), _wmkdir(), etc.)

... but there is still one limitation to this: ZIP and GZ support use zlib, and JMA support uses libjma. Neither of these libraries convert UTF-8 strings to UTF-16 before attempting to open files. Due to licensing issues, as well as technical issues, I am unable to correct this at this time. What this means is that loading ZIP, GZ and JMA files; on Windows only; and with Unicode characters in the file name only; will cause the image load to fail. Loading uncompressed images (SMC, SFC, etc) will work with or without Unicode on all platforms.

I tried to be as thorough as possible with this fix: command-line arguments (via CommandLineToArvW + GetCommandLineW), user path (via SHGetFolderPathW), real path (via _wfullpath),folder creation (via _wmkdir) and file access/existence checks (via _wfopen) were updated in all cases. I also updated file loading for ROMs (SMC, SFC, etc), save RAM (SRM), real-time clock save (RTC), cheat files (CHT), UPS patches (UPS) and both configuration files (bsnes.cfg and locale.cfg.) Configuration file loading should work even if your username contains non-ANSI characters, and it should also detect config files put in the same folder as the bsnes executable, even if the path to the executable contains non-ANSI characters.

Still, if you spot any bugs, aside from the ZIP/GZ/JMA loading issue, please let me know via e-mail at setsunakun0; at hotmail.

Lastly, I'd like to apologize for the poor support for non-ANSI filenames in the past. Using an English version of Windows didn't expose the problems to me. I'll be more thorough in the future with this.


Sun Sep 14, 2008 11:49 pm
has wat u liek
User avatar

Joined: Tue Sep 11, 2007 2:54 pm
Posts: 559
Location: Banland.
Post 
Nice job with the hiro fixes. :)


Mon Sep 15, 2008 2:23 am
Profile
NO VOWELS >:[
User avatar

Joined: Sun Aug 31, 2008 12:59 pm
Posts: 753
Location: ALABAMA
Post 
Fantastic!
Just built last version today in Sid, I enjoy how simple it is to build and install.


Mon Sep 15, 2008 12:24 pm
Profile YIM WWW
Rookie

Joined: Sat Aug 07, 2004 7:20 pm
Posts: 46
Post 
Good job, as always. So, what's next? Time to simulate those peripherals? :)


Mon Sep 15, 2008 5:41 pm
Profile
Veteran

Joined: Wed Aug 04, 2004 5:43 pm
Posts: 861
Location: Sloop
Post 
wertigon wrote:
Good job, as always. So, what's next? Time to simulate those peripherals? :)


Byuu posted in the FAQ thread explaining why this is not happening.


Mon Sep 15, 2008 5:57 pm
Profile
Regular
User avatar

Joined: Tue Mar 07, 2006 10:32 am
Posts: 347
Location: The Netherlands
Post 
Time for sPPU? Though I imagine you might want to get those div/mul delays and the joypad polling thing emulated first..


Mon Sep 15, 2008 6:18 pm
Profile
New Member
User avatar

Joined: Wed May 21, 2008 2:02 pm
Posts: 8
Location: St. Petersburg, Russia
Post 
byuu, big thanks for fixing the problem with loading ROMs under non-English Windows :D

_________________
Some things can only be seen when you've lost everything


Mon Sep 15, 2008 8:46 pm
Profile
Post 
Half of our state is out of power right now, I actually only just managed to post v036 before it happened. They're saying up to seven days. Have intermittent wi-fi access with the laptop, but I'm dead in the water for anything else. Including posting a Hungarian localization I said I would :(

Quote:
byuu, big thanks for fixing the problem with loading ROMs under non-English Windows


Sure, thanks for being so patient with me on that.

So, aside from ZIP/JMA support, it works as expected? Good to hear. Was afraid I'd miss something. Tons and tons of file access code all over the emulator.

Also used the file class, which has built-in buffering that's missing on Windows w/fgetc(), etc. I guess bsnes is a good place to stress test that library, heheh.

Quote:
Though I imagine you might want to get those div/mul delays and the joypad polling thing emulated first..


Yeah. Want to get those two first.


Mon Sep 15, 2008 9:10 pm
Post 
I'll carry this discussion over here.

Nach wrote:
I could easily add for Windows to use a wchar_t * / wstring open call which wraps to using _wopen() to get an fd to build upon.

The issue is how to make it work on all kinds of other OSs, and having it wrap around other source classes when not using libstdc++.


Given that both Linux and OS X support UTF-8 filenames, I say we just add #ifdef _WIN32 wrappers around each file open call. For fopen(), use MultiByteToWideChar + _wfopen(), for std::ifstream, use the same as fopen(), and pass the handle to the ifstream constructor.

No reason to explicity support Windows' broken Unicode system. If they need to, they can convert UTF-16 to UTF-8 with WideCharToMultiByte before invoking the libraries.

Of course, the thing is we need to update both zlib and libjma to do this, and I have a feeling updating zlib won't be as fun. To use WCTMB and MBTWC we will have to include the Windows header, which will hit compile time quite a bit.

Maybe I should write a simple convertor for UTF-8<>UTF-16. It doesn't appear to be too complicated ... certainly nothing worth including a big time conversion library over.


Mon Sep 15, 2008 10:52 pm
Regular

Joined: Sat Mar 04, 2006 3:17 pm
Posts: 307
Post 
Who is wrong here
1)zlib and jmalib using suboptimal UTF stuff
OR
2)Bsnes abusing these libraries

In case 1 the libraries need fixing, as this will help everyone, in case 2 bsnes should have the converter built in.


Tue Sep 16, 2008 9:40 am
Profile
Post 
The answer is, of course,

3) Microsoft was stupid enough to completely duplicate ~80% of their API to take UTF-16 strings, when they could've used UTF-8 with no changes to the underlying API. I know, they were worried about older multi-lingual OSes using non-Unicode codepages. Still a really bad move, NT would've been a good break to force UTF-8.

The best part is that UTF-16 doesn't even allow direct O(1) indexing, as there are extended symbols requiring more than 16-bits to encode.

---

Anyway, the libraries both take const char* filenames, so there's no way I can encode the names for them to work at this time. But it's really not their fault when const char* is good enough for every other OS out there.


Tue Sep 16, 2008 11:00 am
Hazed

Joined: Mon Dec 24, 2007 2:03 am
Posts: 51
Location: Russia
Post 
Well, i do use English, German, Japanese and Russian symbols in filenames. Sometimes they got mixed in to single string, esp multimedia files to note the ones. Those UTF-8 nationalist colored programs always works in twists for me. :(. Most time i'm forced to use pure English and Russian. Really i don't care of bsnes is non unicode program just because most emulators remains non unicode too. It's not comfortable sometimes but i can live with it, np.

_________________
quake2xp audio engineer


Tue Sep 16, 2008 12:36 pm
Profile WWW
Post 
Quote:
Really i don't care of bsnes is non unicode program just because most emulators remains non unicode too. It's not comfortable sometimes but i can live with it, np.


Yeah, noticed that myself when I made the Japanese test profile. Firefox wouldn't start at all, Winamp just kept looping through the setup, but would never actually open or play music. Winrar flat out ignored anything with Unicode in it, and spit out ???s all over. Even notepad2 wouldn't open any text files.

The only things that worked were the ones that came with Windows. Literally nothing else. It was completely unusable like that.


Tue Sep 16, 2008 1:17 pm
Regular

Joined: Sat Mar 04, 2006 3:17 pm
Posts: 307
Post 
byuu wrote:
Quote:
Really i don't care of bsnes is non unicode program just because most emulators remains non unicode too. It's not comfortable sometimes but i can live with it, np.


Yeah, noticed that myself when I made the Japanese test profile. Firefox wouldn't start at all, Winamp just kept looping through the setup, but would never actually open or play music. Winrar flat out ignored anything with Unicode in it, and spit out ???s all over. Even notepad2 wouldn't open any text files.

The only things that worked were the ones that came with Windows. Literally nothing else. It was completely unusable like that.


Yep thats how it is basically.
I have about 2000 files named with different pagetables and they all show up as "$%^TYG$"

But when i install chinese windows in vmware mount that same directory and browse the files they all work.


Tue Sep 16, 2008 1:45 pm
Profile
Hazed

Joined: Tue Feb 07, 2006 11:02 am
Posts: 76
Post 
byuu wrote:
The answer is, of course,

3) Microsoft was stupid enough to completely duplicate ~80% of their API to take UTF-16 strings, when they could've used UTF-8 with no changes to the underlying API. I know, they were worried about older multi-lingual OSes using non-Unicode codepages. Still a really bad move, NT would've been a good break to force UTF-8.

NT was their good break to force Unicode instead of having different code-pages everywhere. According to Wikipedia, the first version of Unicode was released in October 1991 and Windows NT 3.1 was released in July 1993. UTF-8 was first publically announced in January 1993, but it wasn't invented by the official Unicode people and wasn't included in the Unicode standard until Unicode 2.0 in 1996.

Microsoft *could* have maybe pushed the release of NT back a year or so if they'd paid attention to the introduction of UTF-8 and realised how much better this third-party encoding would be than any of the official Unicode encodings, but they didn't.

Quote:
The best part is that UTF-16 doesn't even allow direct O(1) indexing, as there are extended symbols requiring more than 16-bits to encode.

Up until Unicode 3.0 (1999), you *could* fit every Unicode character into a 16-bit integer, so it wasn't such a crazy thing to do in 1991.


Tue Sep 16, 2008 2:33 pm
Profile
Regular

Joined: Sat Mar 04, 2006 3:17 pm
Posts: 307
Post 
So if we had a choice.

Which version of UTF is the best one?


Tue Sep 16, 2008 2:40 pm
Profile
Seen it all
User avatar

Joined: Mon Jan 03, 2005 5:04 pm
Posts: 2302
Location: Germany
Post 
UTF-8.

_________________
vSNES | Delphi 10 BPLs
bsnes launcher with recent files list


Tue Sep 16, 2008 3:03 pm
Profile WWW
Regular

Joined: Sat Mar 04, 2006 3:17 pm
Posts: 307
Post 
Does anyone know of a program that can convert "any codepage" to unicode? (So i can fix my thousands of files that display garbage)


Tue Sep 16, 2008 3:25 pm
Profile
Post 
Posted a new WIP.

The biggest change was that I rewrote nearly all of the cheat code system, so heavy testing on that would be appreciated.

Someone was mentioning over at Snes9X that it was limited to 300 cheats or something, so someone bumped it to 3,000. Not to be outdone (v036 is limited to 1,024), I vectorized the cheat table, meaning you can have infinite cheats now (limited only to available memory.) Actually cleans up the code quite a bit, too. Removed all the ugly strlcpy() stuff, the limitations on description text length, etc.

Looks like I had a bug with deleting codes, too. I wasn't copying the actual cheat codes. That would corrupt the descriptions on every code after the one you deleted, I think. Strange nobody caught that.

I also cleaned up the OBC-1 code, and added a "Fullscreen" checkbox after "Correct Aspect Ratio". Sorry for the delay with that, FitzRoy. Hopefully the checkbox is good enough for now, as I can't change the text to "Switch to ..." just yet.


Tue Sep 16, 2008 3:28 pm
Regular

Joined: Sat Mar 04, 2006 3:17 pm
Posts: 307
Post 
Does the cleanup of the OBC-1 code have any effect on its speed or accuracy?

Cool work on the cheat-engine. I never use cheats so i wouldn't have run into that problem


Tue Sep 16, 2008 3:42 pm
Profile
Veteran

Joined: Wed Aug 04, 2004 5:43 pm
Posts: 861
Location: Sloop
Post 
byuu wrote:
Sorry for the delay with that, FitzRoy. Hopefully the checkbox is good enough for now, as I can't change the text to "Switch to ..." just yet.


Thanks, however, I think modes should operate as a checkmark choice with both listed a la PAL/NTSC. I don't like menu actions that don't take checkmarks, because they look exactly like an unchecked item (I consider this an oversight in modern gui standards). And to have to dynamically change what it says based on the mode... seems unnecessary.

By the way, do you think you could make the emulation speed settings universal rather than mode-specific? I'm slightly annoyed at having to enable/disable them in both modes as they're not the kind of things I would want in one but not the other.


Tue Sep 16, 2008 4:19 pm
Profile
Veteran

Joined: Wed Aug 04, 2004 5:43 pm
Posts: 861
Location: Sloop
Post 
Thinking about this some more, it doesn't really have anything to do with video settings, it's just the way the program is framing it. This looks like an appropriate time to add a new category:

Display Mode >
[x] Windowed
[ ] Fullscreen
---------------
Video Resolution >
[ ] Scale 1x
...

Everything under Video Mode deals with resolution and how it is stretched, so I think renaming that to avoid two modes would be nicer, too.


Wed Sep 17, 2008 1:23 am
Profile
Trooper
User avatar

Joined: Mon Feb 20, 2006 3:11 am
Posts: 394
Location: Space
Post 
So far so good cheat wise. Testing several games and no problems whatsoever. :)

_________________
Kega Fusion Supporter | bsnes Supporter | Regen Supporter


Wed Sep 17, 2008 2:20 am
Profile
Post 
Hello everyone,
Just thought I should mention that the video frameskip option in the windows port does not save when we set it. So that we have to set it every time we start the emu.


Wed Sep 17, 2008 8:06 pm
Regular

Joined: Sat Mar 04, 2006 3:17 pm
Posts: 307
Post 
ssbastos wrote:
Hello everyone,
Just thought I should mention that the video frameskip option in the windows port does not save when we set it. So that we have to set it every time we start the emu.


Yeah, i didnt report it because it thought it might be on purpose


Thu Sep 18, 2008 8:52 pm
Profile
Display posts from previous:  Sort by  
This topic is locked, you cannot edit posts or make further replies.   [ 283 posts ]  Go to page 1, 2, 3, 4, 5 ... 12  Next

Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software.