byuu.org wrote:This release adds SPC7110 emulation, without the need for graphics packs!!, and a rewritten S-RTC (real-time clock) emulator.
SPC7110 support means that Far East of Eden Zero, FEoEZ: Shounen Jump Edition, Momotarou Dentetsu Happy and Super Power League 4 are now all fully playable. I will warn you, the emulation is very slow in this version -- while most areas of each game will run at the same speed as other games, there are a few peak moments where speed will drop by up to ~50%. The reason for the slow-down is that I am currently uncertain how to determine the amount of data to decompress in advance, so I default to the maximum amount possible. The reason I am releasing now anyway, is because I beleive in the "release early, release often" paradigm. It will likely take me a few weeks to finish researching this chip, and I didn't want to keep the work I had private during that time. But rest assured, bsnes v034 should feature much faster SPC7110 emulation.
neviksti, Andreas Naive and jolly_codger worked non-stop on the SPC7110 decompression algorithm for the past two weeks. caitsith2 provided valuable data to the effort. I only wish that I could've been of some use, but alas, I had no role in this. In the end, it was neviksti who managed to crack all three(!!) compression modes of this chip, which turned out to be a customized 8-bit QM-coder with a prediction model. You can read more about this here. I would also like to thank Dark Force and John Weidman (aka The Dumper) for their research notes on the SPC7110 register interface.
For those who don't understand the hoopla about figuring out this compression algorithm when we already had graphics pack simulation, I should note that we have since found a few errors in these packs. Not to mention, you no longer need ~4-16MB packs for each game you wish to run. They work like any other game now. Better still, the chip can now be used to compress new graphics, eg for any future translation efforts on these titles.
The real-time clocks in both Far East of Eden Zero and Dai Kaijuu Monogatari 2 will now save a ".rtc" file in your save folder, which contains the clock as set by the video game, as well as a timestamp from your computer when the time was last updated. It uses the difference between the saved timestamp and current time to update the time. This allows you to specify any time you like, whereas previously bsnes would just use your computer's current time, ignoring the time you set in-game. It also allows the "round clock by 30 seconds" option in both games to work. I avoided this before because this method makes supporting daylight savings time and such impractical, although I should note that the original hardware did not support DST, either. This method was required to pass the SPC7110 tests, and is overall much more faithful to how the original chips worked.
Once again, I'd really like to personally thank neviksti for his tireless efforts. Eliminating graphics packs from SNES emulation was one of my primary reasons for getting involved in the SNES emulation scene. That neviksti managed to crack this algorithm means a lot to me. Thank you so much, neviksti. This release is dedicated to you, now go get some sleep ;)
bsnes v0.033 released
bsnes v0.033 released
-
- -Burninated-
- Posts: 871
- Joined: Mon Sep 10, 2007 11:33 pm
- Location: Unspecified
Damn, many thanks to neviksti and Andreas, and byuu for adding it so quickly to bsnes. I'm amazed at how quickly they managed to crack it. All that experience and knowledge got pooled, we're really lucky to still have them around.
It's funny, I pulled the bounty to fund decapping stuff instead around the time I started the "special chip" thread, totally unaware that byuu had a thread on nesdev trying to generate interest. I don't know if that was a carrot for them, but they're each entitled to a portion if they want.
The only real issue I've noticed so far is that this decompression is a heck of a lot more taxing than S-DD1. I get some pretty massive drops, 30fps on SPL4's title screen for example. Room for optimization or nature of the beast?
Again, awesome work.
It's funny, I pulled the bounty to fund decapping stuff instead around the time I started the "special chip" thread, totally unaware that byuu had a thread on nesdev trying to generate interest. I don't know if that was a carrot for them, but they're each entitled to a portion if they want.
The only real issue I've noticed so far is that this decompression is a heck of a lot more taxing than S-DD1. I get some pretty massive drops, 30fps on SPL4's title screen for example. Room for optimization or nature of the beast?
Again, awesome work.
Lots of room for optimization.
The chip has a length / decrement register, but it does no good since you can read forever from the decompression port. Many times, games do not set the register at all, so you can't use that.
neviksti's decompression routines take fixed sizes, eg "give me an input buffer, an output buffer, and the number of bytes you want decompressed."
Right now, I force it to decompress 32,768 bytes. Even if it was only trying to decompress ~1,000.
So, one of two things need to happen:
One, we need to figure out a way to determine the lengths needed for each table entry.
Two, we need to modify the decoder to a state machine that spits out one byte at a time, and can be reset. This will cause more overhead per byte, but there will be less unneeded decoding, so it should make it significantly faster overall.
I'm leaning toward the latter, but I want to wait until neviksti is 100% done tweaking his decompression code before I try converting it to a state machine.
Another optimization will be making it read directly from the ROM, rather than my memory::cartrom -> SPC7110Codec::buffer transfer, so that neviksti's routines can access the data in a uint8_t array.
Also, this chip has an index into the decompressed buffer that you can specify. The only way this makes sense is if the chip slowly decodes up to that point with the ready flag clear. Whereas again, special chips are treated like memory devices with no clock, so we have to be ready immediately. I don't have a way around that, but games very seldom use the decompression index register anyway. I've only seen it in SPL4 on a dozen or so graphics.
The S-DD1, by comparison, set the DMA transfer registers to the requested output length, so there was never any redundant decompression.
Sorry, I knew it'd be unplayable in some cases, especially SPL4. But I didn't want to sit on this. It always irked me in the '90s when I'd see new special chips being added with teaser pictures, but no release for another six months. I figured something is better than nothing.
The chip has a length / decrement register, but it does no good since you can read forever from the decompression port. Many times, games do not set the register at all, so you can't use that.
neviksti's decompression routines take fixed sizes, eg "give me an input buffer, an output buffer, and the number of bytes you want decompressed."
Right now, I force it to decompress 32,768 bytes. Even if it was only trying to decompress ~1,000.
So, one of two things need to happen:
One, we need to figure out a way to determine the lengths needed for each table entry.
Two, we need to modify the decoder to a state machine that spits out one byte at a time, and can be reset. This will cause more overhead per byte, but there will be less unneeded decoding, so it should make it significantly faster overall.
I'm leaning toward the latter, but I want to wait until neviksti is 100% done tweaking his decompression code before I try converting it to a state machine.
Another optimization will be making it read directly from the ROM, rather than my memory::cartrom -> SPC7110Codec::buffer transfer, so that neviksti's routines can access the data in a uint8_t array.
Also, this chip has an index into the decompressed buffer that you can specify. The only way this makes sense is if the chip slowly decodes up to that point with the ready flag clear. Whereas again, special chips are treated like memory devices with no clock, so we have to be ready immediately. I don't have a way around that, but games very seldom use the decompression index register anyway. I've only seen it in SPL4 on a dozen or so graphics.
The S-DD1, by comparison, set the DMA transfer registers to the requested output length, so there was never any redundant decompression.
Sorry, I knew it'd be unplayable in some cases, especially SPL4. But I didn't want to sit on this. It always irked me in the '90s when I'd see new special chips being added with teaser pictures, but no release for another six months. I figured something is better than nothing.
-
- Locksmith of Hyrule
- Posts: 3634
- Joined: Sun Aug 08, 2004 7:49 am
- Location: 255.255.255.255
- Contact:
maybe try forcing NTSC? Or "unplugging" peripherals (sp) from controller port 2?
That said, congrats all.
That said, congrats all.
<Nach> so why don't the two of you get your own room and leave us alone with this stupidity of yours?
NSRT here.
NSRT here.
To start a game, you have to press 'B', and all the tests will run. Then reset, then press 'A', and the save RAM test(s) will run. Then reset, and the game will start. You only have to do each test one time. Also, you can technically skip the first test on MDH and SPL4, but not on FEoEZ.
And give all thanks to neviksti, please :)
And give all thanks to neviksti, please :)
Thanks everyone. It's nice to see all this outpouring of support.I.S.T. wrote:WOW!
Congratulations on all of your hard work, neviksti, Andreas Naive, jolly_codger and caitsith2!
Congratulations to byuu for releasing it in a form non-coders can use!
Just as a note, pretty much the only reason I wrote the initial code with the last few bits figured out was because I happenned to have more free time than Andreas. That's one smart fella. I wouldn't have been able to do this at all without him, and wouldn't have been able to even keep up with him without the extra free time.
My main "contribution" to the team was more along the "do runs on real hardware" line. Although I should post a picture of the cart modified to have a FIFO instead of a DataROM... it is hideous (it was made to work and nothing else ).
I still don't know how everyone managed to gather there. I hadn't checked the nesdev forum in months and when I did it just happenned to be at the right time. I wonder if it was the same for everyone else.FitzRoy wrote:Damn, many thanks to neviksti and Andreas, and byuu for adding it so quickly to bsnes. I'm amazed at how quickly they managed to crack it. All that experience and knowledge got pooled, we're really lucky to still have them around.
Nah, I just do this for fun (my roommate keeps joking that the amount of time I spend staring at hex code on the SNES vastly outweighs the time I spend playing games on it).FitzRoy wrote:It's funny, I pulled the bounty to fund decapping stuff instead around the time I started the "special chip" thread, totally unaware that byuu had a thread on nesdev trying to generate interest. I don't know if that was a carrot for them, but they're each entitled to a portion if they want.
It would be awesome to see the DSP1b program rom finally extracted though. However I still remain skeptical that this can be done for several hundred dollars... every place I talked to kept quoting me much much more than that. By all means don't let my skepticism stop you or anyone else... just make sure it is more than decapping+pictures that you will be getting before you fork over money, as I've already decapped and taken pictures myself.
-
- ZSNES Developer
- Posts: 6747
- Joined: Tue Dec 28, 2004 6:47 am
-
- ZSNES Shake Shake Prinny
- Posts: 5632
- Joined: Wed Jul 28, 2004 4:15 pm
- Location: PAL50, dood !
Code: Select all
<Nach> Muahahaha!
<Nach> I RE'd a new register not covered properly in our existing docs
<Nach> and now I get the proper length to decompress :D
Code: Select all
<Nach> Okay, I got it up to 60/60 everywhere :D
Last edited by grinvader on Sun Jul 20, 2008 3:07 pm, edited 1 time in total.
皆黙って俺について来い!!
Pantheon: Gideon Zhi | CaitSith2 | Nach | kode54
Code: Select all
<jmr> bsnes has the most accurate wiki page but it takes forever to load (or something)
-
- Regular
- Posts: 347
- Joined: Tue Mar 07, 2006 10:32 am
- Location: The Netherlands
-
- Rookie
- Posts: 32
- Joined: Wed Aug 25, 2004 1:55 pm
- Location: spb, ru
- Contact:
-
- Romhacking God
- Posts: 922
- Joined: Wed Jul 28, 2004 11:27 pm
- Contact:
Cheers to all involved in cracking the chip. Very nice community effort. I've followed the developments, but had nothing helpful to contribute. We've got some intelligent beans amongst our community that I am happy are among us.
[url=http://transcorp.romhacking.net]TransCorp[/url] - Home of the Dual Orb 2, Cho Mahou Tairyku Wozz, and Emerald Dragon SFC/SNES translations.
[url=http://www.romhacking.net]ROMhacking.net[/url] - The central hub of the ROM hacking community.
[url=http://www.romhacking.net]ROMhacking.net[/url] - The central hub of the ROM hacking community.
-
- Hazed
- Posts: 76
- Joined: Sat Jan 28, 2006 7:21 am
Cool, has the team working on the chip been told about the new register. I ask in case one of them wants write a doc fully explaining the chip.grinvader wrote:(...)Code: Select all
<Nach> Muahahaha! <Nach> I RE'd a new register not covered properly in our existing docs <Nach> and now I get the proper length to decompress :D
Code: Select all
<Nach> Okay, I got it up to 60/60 everywhere :D
Nach doesn't even like using a real snes, and last I heard doesn't even own one. So take claims of "reverse engineering" an unknown/new hardware register lightly.bobthebuilder wrote:Cool, has the team working on the chip been told about the new register. I ask in case one of them wants write a doc fully explaining the chip.
What byuu said above is correct. What Nach said should probably be better interpretted as he "found some hacks to put in the emulation to speed up these games".
-
- ZSNES Developer
- Posts: 3904
- Joined: Tue Jul 27, 2004 10:54 pm
- Location: Solar powered park bench
- Contact:
I'd appreciate if you don't make stuff up about me.neviksti wrote:Nach doesn't even like using a real snes, and last I heard doesn't even own one. So take claims of "reverse engineering" an unknown/new hardware register lightly.bobthebuilder wrote:Cool, has the team working on the chip been told about the new register. I ask in case one of them wants write a doc fully explaining the chip.
What byuu said above is correct. What Nach said should probably be better interpretted as he "found some hacks to put in the emulation to speed up these games".
May 9 2007 - NSRT 3.4, now with lots of hashing and even more accurate information! Go download it.
_____________
Insane Coding
_____________
Insane Coding
Your "software" vs. "real hardware" approach came up multiple times in discussion after byuu released his "state of emulation" rant. It was there that you made such comments. If they have changed, so be it. I may have misconstrued your comments about owning a snes at the time, but that was my recollection from what you said. I am not just making stuff up.Nach wrote:I'd appreciate if you don't make stuff up about me.
And if you have discovered a "new" register, please do let us know. I have run tests on the real hardware and it appears that what byuu said is correct "The chip has a length / decrement register, but it does no good since you can read forever from the decompression port. Many times, games do not set the register at all, so you can't use that." The only other registers you need to write to in oder to start a decompression cannot be used to determine a hardware decompression length. So the only option that seems reasonable to me is that you have found some emulation hacks specific to these games (probably DMA related)... and not found a new register. Of course, I could be wrong. New registers are exciting finds... please do share.