ToonPal on USF by ToonPal at 11:53 AM EST on February 28, 2006
PSF is a pretty impressive invention, being able to take just the minimum code and data for a piece of music from a game for any supported system, pack it into a single file, and play through a program/plugin that simply emulates only the necessary console hardware to play back the tune on a reasonably-equipped desktop computer.
USF is no less impressive, although from what I've learned from trying out all the different tunes that have been ripped, it's been a somewhat silly yet educational experience.
Before emulation, I thought that the N64 sound system was comparable to that of the Sony Playstation. Man, was I surprised at how wrong that assumption was! The audio hardware behavior now reminds me of a Sound Blaster 16! I'm saddened to see how the system is forced to make the little 90MHz CPU do the sound mixing work AND run the rest of the game at the same time! I was surprised when I saw so many rips getting played back at mixing rates of 32 kHz and less! Getting as low as 22 from what I've tried out so far! Those poor game developers! Having seen the Konami Goemon game soundtracks press forward with full 44kHz soundtrack leaves me extremely impressed! It sounded like they had to avoid the use of reverb and stuff to make it work without taxing the CPU, but it still sounded great! Alright, enough exclamation points! ...points.
It's pretty sad to see the memory allocated to play back just one song being equal to that of the full size of the ROM from which it was ripped and also the CPU time necessary to emulate an N64 CPU running at near-half-load to play the music right. I have a pretty good idea of why these things must be done, and I bet all you emulator developers also think it's sad that such extremes must be taken to make it work.
I bet some of you also think, "Dear Lord! What ever happened to the simpler times of gaming sound system emulation like that of the SNES SPC!?" :>
The main CPU generally is not used to do the sound mixing, the RSP does that. The exception is a few games that do minimal synthesis (F-Zero X, for example), because they want to save RSP time for 3D processing (or they just have a lot of ROM to spare).
The memory allocation for the ROM should be less that the total ROM size, as it only allocates 1 MB chunks as needed. The dynarec CPU uses no small amount of memory, though.
Wow! You learn something new everyday! RSP, huh? I guess that still means that it's impossible to render the music using native code like for SNES SPC and PSX SPU players. (If that assumption is wrong as well, please correct me! I don't feel silly enough with my educated guess about the N64 CPU core.)
It's interesting what you mentioned about the dynamic recompiler and memory usage. I decided to play with that just for fun. I fired up Winamp and started playing songs from various games with the Recompiler CPU turned off. Memory usage did go down a little bit, but not as much as I thought you meant. In numerous Goemon's Great Adventure tracks, free memory dropped by 16 megs. In Super Super Smash Bros., about 14. In ExciteBike 64, 16. In F-Zero X, my jaw dropped as numerous tracks caused memory levels to drop by 24 megs! With stuff like this going on, I was assuming that the emulator was having to fully load a modified ROM image containing about a meg of sound code and data and the rest being zeroes. Note that this is with the Recompiler CPU turned OFF and CPU usage superbly increased, which I assume means that the plugin is falling back to an Interpreter CPU.
All it's doing is playing music! What the heck!?
I guess I could also tell you my software configuration while I'm at it. I'm running Winamp 5.08e and 64th Note v1.1 beta 4 on top of Windows 98 SE.
It's an honor to exchange techno-babble with such a talented programmer!
It is not impossible to render the music with native code, but much more difficult to get right. Audio HLE is the method used to do this, but it is a simulation of software, not standardized hardware as older consoles had. As such there are many games which are improperly supported, because they use differing code for music generation. The RSP recompiler (and CPU recompiler, to a lesser extent) is an attempt to overcome this by generating native code to represent the synthesis software at runtime. You don't even want to know how slow the RSP is in interpreter mode (and it isn't included as an option in 64th Note because the recompiler works so well, props to zilmar and jabo).
It's interesting that you mention that the RSP is always in recompiler mode even when the CPU is not. It's actually kind of confusing. If the RSP is doing most of the work, why does disabling the CPU dynarec increase host CPU usage so much? Oh! I bet it's because the CPU's cycles need to be emulated to keep the RSP going! or... not... I'll let you answer the questions if you're willing. This is all incredibly interesting!
Even though the RSP is doing most of the brute number crunching, the CPU is still running a multithreaded OS. If the rips were done better this would be less of an issue, but USF is designed around a lazy ripping strategy.
The audio microcode context-switches the RSP away from it's graphics work at relatively unpredictable times during the frame, so depending on where your bottlenecks lie it could be advantageous to mix on the main CPU. One game I know of runs a stripped down SPC700 emulation on the main CPU (complete with BRR samples) and that lets it keep the RSP pumping out geometry.
Now, Nintendo *did* learn from this: the DS, which otherwise is relatively comparable to the N64 (but with a far more conventional 3D pipeline), has a very nice 16-channel synthesizer chip that can play 4 different sample formats, so it's more or less up to PS1 specs there. I'm sorta hoping someone figures out a way to do music rips of DS games without actually emulating the system.
Likewise the Gamecube has a dedicated audio DSP, which has "ucode" built in for the purpose of decoding the standard ADPCM format Nintendo provides, though it is reprogrammable.
There's a program (I don't remember what it's called, sorry) that can read DS ROMs, and output MIDI files. From what I've heard, it works in most cases. So, ripping the sequence data sounds like it could be pretty easy. Ripping the sample banks, and pairing up samples to the MIDI properly, could prove more difficult (I don't know that, I'm just saying that, since it rips MIDIs but no samples, that it seems getting the samples is more difficult). I know there's a few people on #usf that use it. maybe I'll be able to come back later, and tell you the name of this program. Hopefully that helps in some way. Mouser X over and out.
I like using the gba2midi and ssequtil for my own use (and no I do not submit them to vgmusic.com because it isn't my work and that they don't allow it anyway). Actually I just like listening to midi on XG drive. It does take time to "fix" the output midi because they're not in the standard midi bank. You can hear how it turns out but it requires a XG drive on your computer because I loaded them using XG banks and such. :)
But still I would like to see some sort of DS sequenced player in the future.