SNES NTSC Composite Video Filter

[SteveSnake]

(the PPU waits an extra pixel clock every other frame)

Ah, OK, I thought this was happening at the TV end and not the SNES end. Fair enough

I'm making steady progress and would like to help make a version for the Sega Genesis, if you're interested.

Indeed I am. Since many people seem to want this already, I was planning on talking to you about it at some point

(it's interesting)

Certainly is. Another thing that might be interesting - do you currently reproduce the colors in the same way as NTSC does? Because NTSC-J looks *very* different. Would be cool to be able to handle that too

[vkamicht]

Hmm, so this is supposed to 'emulate' how a TV would look?

So if I were going to use my vid cards video output to a TV I would want to leave this filter off?

[kick]

It would be great if ZSNES automatically sets VSYNC ON and switches display refresh to 120Hz or 60 Hz when this filter is active.

But what about PAL games? Is it kinda odd when running PAL roms with this filter enabled? There needs to be another filter for PAL as well.PAL video looks (colors,gamma,blurring) different from NTSC.

blargg,if you're working on the Genesis filter,will you add the "vertical lines" simulation of the Genesis display output?

I can't wait for the update of Nestopia's NTSC simulation filter.It currently uses so much CPU time,it's nearly unuseable.

[blargg]

Hmm, so this is supposed to 'emulate' how a TV would look? So if I were going to use my vid cards video output to a TV I would want to leave this filter off?

It emulates both the artifacts inherent in the SNES NTSC composite output circuitry and the way an NTSC TV decodes the signal. TV output cards vary; my guess is that you won't see much difference (assuming that the other ZSNES filters display with the same aspect ratio as my NTSC filter). If you have a SNES you can find out which looks closer to the real thing and post your result.

It would be great if ZSNES automatically sets VSYNC ON and switches display refresh to 120Hz or 60 Hz when this filter is active.

I'm sure that once the NTSC filter is in a general release, it will have a much more streamlined interface and operation. As I understand, the current version was put up to basic feedback and to get something out now, rather than waiting for it to be fully integrated and polished.

But what about PAL games?

I'm glad you asked. My only aim with this filter is NTSC composite video emulation (that is, US televisions). I haven't studied PAL and never even seen a PAL television, so I'm not currently able to do much for a PAL filter. I think I'll be able to find a few people who are more knowledgable when I'm ready for it. I should be receiving a PAL NES soon and fortunately my video capture card can handle PAL, which will help the process along.

As for Sega Genesis, if you mean what I think you mean, the vertical column distortion on pixels will surely be emulated. And I've just e-mailed the Nestopia author my latest NES NTSC filter code that is based on the same improved code used by this SNES NTSC filter. I've included a way for him to write his own optimized versions of the speed-critical graphics loop, so he should be able to get it running quite fast. I'll eventually release the same for the SNES version, hopefully so the zsnes team can write the core loop in assembly and bump their asm percentage up a bit.

Since these share the same basic codebase, feedback for a specific version usually benefits every system. I'm excited about how much this has taken off since my latest speed breakthrough, and wouldn't be surprised if I end up making versions of this for a dozen systems or more.

[Nightcrawler]

It emulates both the artifacts inherent in the SNES NTSC composite output circuitry and the way an NTSC TV decodes the signal.

You mind expanding on this. Exactly what artifacts does the SNES output circuitry produce? I can handle a technical explanation too.

[blargg]

Until I write a complete description, I'll only summarize. NTSC represents luma (brightness basically) by the amplitude of the video signal. It represents chroma (color) by adding a ~3.58 MHz waveform to the luma signal, whose phase is the hue and amplitude is the saturation. In the TV the signal is filtered back into the luma (lower frequency) and chroma (the region around 3.58 MHz). SNES pixels are generated at a rate of ~5.37 MHz, or ~186 nanoseconds (ns) per pixel. If you have a single pixel of some color with black around it, there's not enough time to establish a complete cycle of the color waveform (~279 ns per cycle), so the TV decoder sees something at a different phase and amplitude than the SNES intends. ASCII art, engage!

Code: Select all

  ,-~-,
 /     \
/       \       /
         \     /
          '-_-'
|----------------| 279 ns

One color cycle

  ,-~-,
 /     \
/       \ ,-------
         \|
          '
|---------| 186 ns

One SNES pixel possibility

                  ,-.
                 /  |
----------,     /   '--
          |    /
          '-_-'
          |---------| 186 ns
          
Another SNES pixel possibility

If you put multiple SNES pixels of the same color together, you get enough of the color wave that the color is properly represented. You get artifacts even for lone gray/white pixels because they are narrow enough to contain significant harmonics in the color range.

It's just not possible to represent the color of a pixel narrower than around a color cycle, nor to switch colors faster than that, due to the inherent amount of information that can be encoded.

[pagefault]

Ok I am trying to implement the high res version of it but I have little knowledge how the SNES renders a pseudo hi-res mode frame. The filter takes in a 512x223 buffer as I understand it but I am unsure how this is supposed to be rendered from the video frame buffer. Does anyone have any info on how these high res modes are supposed to be rendered?

[creaothceann]

Well, there's a bit of info in this document.

I'm combining the main- and subscreen pixels to a normal 256x? screen, but that's just a convenient hack...

[byuu]

Pseudo-hires and true hires are nearly identical. To add pseudo-hires though, you must first emulate hires correctly.
That means you render the screen internally as 256 pixels, still. The "even" pixels from the 512-width frame go into the main pixel, and the "odd" go into the subpixel. Then at the end of the scanline, you process the color math slightly differently to generate a unique pixel for both the main and sub screen pixels, hence doubling the width to 512.
Pseudo-hires is the same thing, only it doesn't render two pixels from each BG layer at a time, it only renders one. And that pixel can be rendered to the main pixel, sub pixel, or both. So to make a 512-width image in pseudo-hires mode, you would need all even pixels on BG1, with BG1 only having the main screen enabled, and all odd pixels on BG2, with BG2 only having the sub screen enabled. At this point, you have the full 256-width scanline, and rendering is from this point on 100% identical to hires mode.
The only difference between hires and pseudo-hires mode is that pseudo doesn't grab two pixels per "dot", hence tiles are still 8x8 instead of "16x8".

Pseudo-hires usually uses two separate images on the different background layers, and this results in the two blending together, hence why the 50% average trick works somewhat, whereas hires is usually used for more detailed fonts, and this is why simply blending the two here is a bad idea. But the reality is, the effect is the same for both. You could just as easily blend two images in true hires, or draw a high-resolution font in pseudo-hires. You'd just have to change the way the data was stored and backgrounds were used.

Basically, as far as rendering goes, the theory is that modes 5 and 6 basically ignore the pseudo-hires bit and act as though it's always enabled, whereas the effect is optional in modes 0-4+7. So yeah, even mode 7 can use 512-width mode.

[blargg]

I wrote a bit about how the filter works:

A SNES pixel's effect is spread out on the TV screen, due to the way NTSC encoding and decoding work. At some point I realized that the NTSC operation was linear, which meant that the result of running two adjacent pixels A and B through the NTSC filter is the same as running A and black through the decoder, black and B through separately, then summing the two outputs. This opened the door to precalculating each possible pixel appearance in isolation and then combining them when running the filter.

For each possible SNES color, a single pixel of it on a black background is generated at each combination of the three column alignments and three scanline burst phases, totaling 9 possible appearances (see earlier post about the burst phases):

Code: Select all

  012 012 012
0 X-- -X- --X
1 --- --- ---
2 --- --- ---

0 --- --- ---
1 X-- -X- --X
2 --- --- ---

0 --- --- ---
1 --- --- ---
2 X-- -X- --X

A SNES pixel affects many output pixels to the left and right, due to filtering in the TV, so for each of the above combinations, the 14 non-zero output pixels are captured into a kernel. An imporant subtlety is that the output pixels are signed RGB values, since a SNES pixel can both increase and decrease the RGB values of nearby output pixels. So the kernel contains the signed red, green, and blue components for each of the 14 output pixels, centered around the original SNES pixel.

There are 32768 SNES colors available. Each kernel uses 56 bytes (14 pixels * 4 bytes per pixel), so if a set of 9 kernels were made for every color, almost 16 megabytes of memory would be needed for the table, and almost 300000 kernels would need to be calculated during initialization. This would take quite a while, since NTSC signal processing is somewhat complex. I found that I could reduce both the red and blue channels from 32 possible intensities to 16 without affecting the image noticeably. This reduced the number of colors to 8192, one fourth of the original.

Finally, when running the filter on a full SNES image, the appropriate kernels for each SNES pixel are summed together. Below, SNES pixels are numbered from 0 upwards, where 0 is the first pixel on the line.

Code: Select all

|----- 0 -----|----- 6 -----|---- 12 -----|------------ ...
  |----- 1 -----|----- 7 -----|---- 13 -----|---------- ...
    |----- 2 -----|----- 8 -----|---- 14 -----|-------- ...
       |----- 3 -----|----- 9 -----|---- 15 -----|----- ...
         |----- 4 -----|---- 10 -----|---- 16 -----|--- ...
+          |----- 5 -----|---- 11 -----|---- 17 -----|- ...
-----------------------------------------------------------
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX ... output pixels
             *

As you can see, any given output pixel depends on the nearby 6 SNES pixels. For example, the starred output pixel is the sum of k0 [13], k1 [11], k2 [9], k3 [6], k4 [4], and k5 [2], where k0-k5 are the kernels for pixels 0-5, and k [0] is the first pixel of a given kernel.

One additional step needs to be performed. The final RGB values can be less than zero or greater than the maximum, so they need to be clamped, otherwise they will wrap around and look bad. Clamping turns negative values into zero and excess values into the maximum. After clamping, the RGB components converted to the packed format used by the graphics card.

It's quite an intensive calculation, even with the table optimization. I put a lot of work into encoding the RGB values into the kernels so that they could be efficiently summed and clamped with a minimum of calculation. Even kernel initialization needed significant optimization to prevent it from taking several seconds. This has come a long way from the original NES NTSC code and algorithm given to me by NewRisingSun, which ran at around 2 frames per second on my machine. The optimized table implementation now runs 75 times faster at 150 frames per second.

[tetsuo55]

First of al i want to thank you for the great NTSC filter!

As i live in pal-land i would love to see a pal version of your filter

unlike NTSC there are no visable scanlines on pal TV's, but the colorbleeding should be about the same

i know you dont have a paltv and no pal systems (yet) i found an interesting document that might be able to help you convert your algorythm to a pal version, all we need to do is disable the scanlines with that new code and it should look a lot like pal

this is the site: http://www.kolumbus.fi/pami1/video/pal_ntsc.html

the best thing about the site it that it shows you all values of pal and ntsc side by side, so you know exactly which values to change for pal output!

another interesting resource is : http://en.wikipedia.org/wiki/Pal, which in turn leads us to a SECAM filter : http://en.wikipedia.org/wiki/SECAM

now all we need is info on the japanese ntsc signal

i found the following on NTSC-J

Only Japan's variant "NTSC-J" is very slightly different: in Japan, black level and blanking level of the signal are identical, as they are in PAL, while in American NTSC, black level is slightly higher than blanking level. Since the difference is quite small, a slight turn of the brightness knob is all that is required to enjoy the "other" variant of NTSC on any set as it is supposed to be; most watchers might not even notice the difference in the first place.

[grinvader]

When a bunch of pixels of the same color are together, the phases average out and you see a solid color without any irregularities. A single pixel of a color on black will show the differences much better. Below are some example pixels on the left and what the appear as on the right on even and odd frames (through my NTSC filter). They are expanded 300% to make them more visible.

Well, Either this phenomenon is not noticeable or it doesn't happen at all with PAL systems. Probably due to the phase inversion every other line.
I'm talking about both old pieces of trash older than me as well as brand-new PAL CRT tvs. You can see the flickering due to the lower refresh on PAL50, but 0 colour additions like that.
Also I play quite close to the TV - close enough to see the PAL scanlines - but can't see any colour warp due to scrolling, either with PAL50 or 60.

[Nightcrawler]

Thanks for the explanation blargg. That cleared up what it is your doing. It's amazing that NTSC has been around as long as it has. I find out more and more NTSC limitations every day almost.

[Lord Nightmare]

Yes, PAL has a much more stable color synchronization than NTSC does, since clock drift on one line is undone on the next line. However, color bleeding still happens due to the inherently limited amount of information that can be stored in the color signal. The color clock runs at a different rate on PAL than on NTSC iirc (I might be wrong) and obviously the refresh rate and number of scanlines is different.

I leave it to others to continue this as I don't know much more.

Lord Nightmare

[SteveSnake]

unlike NTSC there are no visable scanlines on pal TV's

Yes there is. All lines are closer together because they squeezed an extra 100 lines in there. But there are always scanlines because the system uses the even lines only. Look closer.

but the colorbleeding should be about the same

Nope, PAL has virtually no noticable colour problems as such. It is still pretty low res, so you will get a certain amount of blending going on, but the colours themselves look right.

Only Japan's variant "NTSC-J" is very slightly different: in Japan, black level and blanking level of the signal are identical, as they are in PAL, while in American NTSC, black level is slightly higher than blanking level. Since the difference is quite small, a slight turn of the brightness knob is all that is required to enjoy the "other" variant of NTSC on any set as it is supposed to be; most watchers might not even notice the difference in the first place.

Unfortunately that is completely and utterly wrong. The colours of NTSC-J look *VERY* different. If you can't notice the difference immediately then you need new eyes

[Verdauga Greeneyes]

Yes there is. All lines are closer together because they squeezed an extra 100 lines in there. But there are always scanlines because the system uses the even lines only. Look closer.

You could be right, but I really can't see them on this particular TV; either that or they're only half or less the height of the visible pixels. In fact, if anything it looks like there's vertical scanlines, but that's just the blue subpixels.

[grinvader]

There are horizontal scanlines on PAL tvs. They are just smaller than on NTSC sets.

The even and odd fields are both used, but there are still scanlines because they aren't used at the same time, heh (and the lower scan rate produces the flicker).

[tetsuo55]

Well i just quoted over the ntsc-j info, i guess its wrong

never seen an ntsc-j tv in action

about the pal scanlines

i know the scanlines should be there, but i checked it on 10+ different tv's and there are no visable scanlines

once i get close enough i do see vertical lines (im at less than 30 cm distance from the tv now)

but at the recomended distance of 1 mtr or more the image looks just like a monitor but with more flicker and a softer image

thats why (when available) i always use pal60 over the RGB scart port, its a more stable image, no black bars and better color than regular pal50

[Verdauga Greeneyes]

The verticle scanlines are just the blue subpixels, I think, though there are tiny gaps in between the pixels as well of course and I can see those. But no detectable horizontal scanlines.

[Sith]

...thats why (when available) i always use pal60 over the RGB scart port, its a more stable image, no black bars and better color than regular pal50

Ah yes, bless GC PAL60.

I never used DC PAL60 on the other hand because you couldn't correct the offset screen position from the scart image (the larger image causes even more loss at the left edge) & somehow colors & contrast seemed weaker too...

[tetsuo55]

...thats why (when available) i always use pal60 over the RGB scart port, its a more stable image, no black bars and better color than regular pal50

Ah yes, bless GC PAL60.

I never used DC PAL60 on the other hand because you couldn't correct the offset screen position from the scart image (the larger image causes even more loss at the left edge) & somehow colors & contrast seemed weaker too...

wierd about the dc, i couldnt comment on it because my dc is japanese so it displays ntsc

which means pal60 always for me

[blargg]

Well, Either this phenomenon is not noticeable or it doesn't happen at all with PAL systems.

Man, sounds like Europe got the color standard right.

To reiterate, my NTSC filter is only aimed at NTSC (US), not PAL (Europe) or televisions in general.

Yes, PAL has a much more stable color synchronization than NTSC does, since clock drift on one line is undone on the next line.

So that's why they didn't call it "never the same color". The artifacts around pixels is not due to color drift, though, since it happens even when everything is perfectly synchronized. It's due to the inability to encode that much color information into the signal. PAL apparently uses the previous line's color information or something to "fill in the gaps", since usually the previous line has similar color information as the next.

The colours of NTSC-J look *VERY* different. If you can't notice the difference immediately then you need new eyes.

I have the decoder matricies used by some some Sony TVs in Japan, and they are shifted a bit. In the next release of the NTSC filter I'll add the ability to use these. I'd include the black level difference but I don't have anything to compare with to be sure I have it right.

[Lord Nightmare]

So that's why they didn't call it "never the same color". The artifacts around pixels is not due to color drift, though, since it happens even when everything is perfectly synchronized. It's due to the inability to encode that much color information into the signal. PAL apparently uses the previous line's color information or something to "fill in the gaps", since usually the previous line has similar color information as the next.

Erm, not quite. other than the obvious differences of signal timing and number of lines, the only other major difference that PAL has vs NTSC is that the color clock phase is switched at the beginning of each scanline, so that color doesn't get shifted weirdly toward the bottom of the screen by the TV internal color clock getting desynchronized from the color clock in the incoming signal (the colorburst).
Hence, Phase Alternation by Line. PAL.

The color clock shift doesn't help color stability within a scanline any, it just helps prevent TVs with an unstable internal clock from showing really WHACKED colors on the lower portion of the screen as the internal color clock goes up to half a phase off of where it should be. (if it goes more than that, the color shifts slowly back to normal until its completely 1 full phase off) This has something to do with a popular method of generating and inverting the color clock phase, when it was on a 'positive phase' scanline it would drift postitively, but on a 'negative phase' scanline it would drift negatively. Some color clock generators would be unaffected by the necessity of inversion, and hence could display just as badly warped colors as an NTSC TV would.

DISCLAIMER: Most of the information above is from my own reasoning based on what I've read, I could be WAAAY off on some pieces of info, so try to read up and correct me if I made any obvious mistakes.

Lord Nightmare

[Ichinisan]

[awesome information]

[offtopic]You were the best playable character in NBA JAM.[/offtopic]

[Bahamut_ZERO_Clue]

Maybe it's my resolution or my eyes but the NTSC Filter does nothing from the most recent CVS sources. Is my resolution of 1024x768 too high a resolution?