There are a few games for which I'm trying to compile a music tracklist straight from the ROM. For an example of what I'm talking about, see the list I found at ROMhacking.net for Super Mario RPG on SNES that I've included at the bottom of this post.
I'm not trying to rip music exactly--just locating it in the ROM and seeing what the particular game counts as actual music (i.e. not sound effects, not audio for movie files, not every single variation/instrument arrangement of every melody, etc.). And I want to see what order they're arranged in the ROM. (I don't pretend there's some higher purpose for this--I just enjoy seeing how files and things are laid out within game ROMs.)
The main game I'm interested in seeing this for right now is Paper Mario (but I'd also like to look into SSB, OoT, MM, and SM64 at some point). Instead of asking one of you more experienced folks to make such a list for me, I want to ask how I can do this myself? I'm a complete newb when it comes to hacking/digging around inside ROMs, but I want to learn.
The tools I have are the ROM itself, Project 64 (as it's an emulator, I don't know how useful it will be for seeing files inside the ROM), and a hex editor known as HxD (is there a better hex editor I should use?). What else do I need?
For Paper Mario, I downloaded the USF rip from Zophar's Domain. According to the readme on that set from hcs, if I'm reading it right, there are 143 tracks that are considered valid songs. I'd really like to know what those 143 songs are, and what order the game puts them in. What do I need in order to figure this out? Can anyone help me?
Anyway, for a great example of the sort of thing I'm looking for, below is a list of BGM songs for SMRPG that I found at ROMhacking.net (byte reference in one column and track title in the other):
00 Silence
01 Dodo's Coming
02 Mushroom Kingdom
03 Fight Against Stronger Monster
04 Yo'ster Island
05 Seaside Town
06 Fight Against Monsters
07 Pipe Vault
08 Invincible Star
09 Victory
0A In The Flower Garden
0B Bowser's Castle (1st time)
0C Fight Against Bowser
0D Road Is Full Of Dangers
0E Mario's Pad
0F Here's Some Weapons
10 Let's Race
11 Tadpole Pond
12 Rose Town
13 Race Training
14 Shock!
15 Sad Song
16 Midas River
17 Got A Star Piece (part 1)
18 Got A Star Piece (part 2)
19 Fight Against An Armed Boss
1A Forest Maze
1B Dungeon Is Full Of Monsters
1C Let's Play Geno
1D Start Slot Menu
1E Long Long Ago
1F Booster's Tower
20 And My Name's Booster
21 Moleville
22 Star Hill
23 Mountain Railroad
24 Explanation
25 Booster Hill (start)
26 Booster Hill
27 Marrymore
28 New Partner
29 Sunken Ship
2A Still The Road Is Full Of Monsters
2B Silence
2C Sea
2D Heart Beating A Little Faster (part 1)
2E Heart Beating A Little Faster (part 2)
2F Grate Guy's Casino
30 Geno Awakens
31 Celebrational
32 Nimbus Land
33 Monstro Town
34 Toadofsky
35 Silence
36 Happy Adventure, Delighful Adventure
37 World Map
38 Factory
39 Sword Crashes And Stars Scatter
3A Conversation With Culex
3B Fight Against Culex
3C Victory Against Culex
3D Valentina
3E Barrel Volcano
3F Axem Rangers Drop In
40 The End
41 Gate
42 Bowser's Castle (2nd time)
43 Weapons Factory
44 Fight Against Smithy 1
45 Fight Against Smithy 2
46 Ending Part 1
47 Ending Part 2
48 Ending Part 3
49 Ending Part 4
In the more general case, there may be several different paths for setting up sequences, sometimes for different variations, sometimes for totally different sets of songs; the Totakeke live and aircheck songs in Doubutsu no Mori were controlled by different routines, for instance, at least at the level I dug into it, and the winter and cherry blossom festival variations of the music required additional passes through the rip. I'm told that the bonus C64-style songs in Tetrisphere were set up completely differently from the main soundtrack.
If you're lucky you can find gameshark codes that you can play with, Ocarina of Time has a nice one but I'm having some difficulty locating it since gscentral passed away.
If you have a USF rip you might be able to get some help. The .miniusf file format is a patch on top of the main .usflib for a game, and sometimes this patch just consists of specifying a song select value. For instance here is track 82a from ugetab's OoT 1.2 rip:
The only part that's important for the music is the bytes after the SR64 identifier:
04 00 00 00 this is a 4-byte patch,
A4 B9 0B 00 written to address 0xBB9A4 (within the PJ64 save state, I think you subtract 0x75c to get the bare RDRAM offset)
67 00 05 24 value: 0x24050067
00 00 00 00 a 0-byte patch (i.e. no more patches, EOF)
I'm pretty sure that 0067 is the song number for this track (End Credits A), (the rest of that number, the 2405 part, is just whatever happened to be nearby in memory).
You can check the specs for the hard facts on how USFs work.
---
Another point: If you get an untagged USF set, the files are probably named sparse???.miniusf, the number in that name should be the song select number for that song. It may be possible that there are untagged versions of tagged sets floating around. This would be a good deal easier than digging into the hex...
---
Caveat: I wouldn't count on USF rips as giving the canonical listing of all the songs in a game.
Hope that helps.
edited 6:44 PM EDT April 26, 2014
---
Oh, this is kind of cool, I looked into Paper Mario, track 101a looks like this:
You can read that as 4 bytes (00 00 00 95) at 0x70 (which is early in the save state, this is actually a CPU register), 4 bytes (00 00 00 95) at 0xD0 (another register), and 4 bytes (00 00 00 00) at 0x15A260.
0x95 is the song number for this track. That 0 at the end is interesting, it turns out that that is the section control that I was talking about back here. Look at track 101b:
That specifies we're still dealing with sequence 0x95, but now section 1.
edited 7:09 PM EDT April 26, 2014
I once downloaded a file called "gsc_codes_backup-091904"
Seemed like a useful thing to keep around:
gsc_codes_backup-091904.7z
I also have BSFree.org bookmarked — which if I recall was built from that same archive. Their parsing seems broken with the OoT files but they also have a link at the top of their page to gamehacking.org which seems a more actively curated resource.
In my own hex editor, I'm looking at Paper Mario, but I can't find the SR64 identifier? I do a search for it, but nothing comes up. (I couldn't find "PSF!" either.) Did you write those in yourself in your editor? At what hex location in the ROM do these tracks start?
My next issue is going to be playback, so I can hear and identify each track. I have Project 64. How useful would that be? Should I download Gameshark for that?
One idea I had was, if I could locate in the ROM the track that plays when you first start up the game ("Story of the Stolen Spirits" I think), I could simply replace that track with another track and just start the game on an emulator to see what plays during the prologue/backstory, then do that for the subsequent tracks as well. How feasible would this option be? (I'm rather leery of changing any values in the actual ROM!)
Oh yeah, if anyone has any idea where I can find untagged .usf rips of PM, SSB, OoT (1.2), MM, SM64, and MK64, I would be much obliged.
And again, thanks so much for your help so far! (And thank you too, BtEO, for the GSC codes!)
Your idea about replacing the song that plays on startup is good, but I can't be of much help with that beyond what I wrote in the info.txt that comes with the rip.
Thanks for the codes, BtEO, found the one I was looking for, though it turns out to have been for the Master Quest version, LoZ: OoT MQ music mod.
Okay, so I'm using Windhex, and looking at both the USFLIB file from the .USF set as well as the ROM itself, I remain just about as clueless as when I began.
First, the USF set file. I see what you're showing in the screenshot, hcs, but it doesn't make any sense to me. How do I know it's track 101a? How do I know what's 101b? The readme document said there's a song select at 0x80053CF8, but trying to go to that (or 80053CF8) takes me nowhere because that's a higher value than the file actually is (which only goes up to 00453AE0). I read what you said underneath the screenshots, but--forgive me--it was like reading Greek. =(
Next, the game's ROM. I had a bit more success here in that I was able to locate the timestamp as well as BGM indicators. I dumped the binary data and indeed found all 143 instances of BGM. Each instance seems to be punctuated by a three-digit number, seeming to begin with 117 but skipping around to 191, etc. I don't know what any of that means, though. Do they in any way correspond to the track listing in the USF set? I wouldn't think so.
So basically I'm no closer to finding out what the 143 songs are nor which order they're in. (I mean, if somebody already knows and they just want to post the list here, that would be awesome.) I just need to apply what I found in the ROM. Is there any way I can use Project 64 for this purpose? My earlier idea of changing the track for the prologue screen now seems like a bad idea (after looking at WindHex, I have no idea what value I would change nor to what). I also don't know how I would use Gameshark for Paper Mario, nor if I would need to (I don't even have it, so I'd have to download it and everything).
So can anyone help? Sorry if I'm being a bother.
You can tell what song that is by playing it in 64th Note or some other USF player, or just by recognizing the track name, or if you can find the soundtrack you could look it up there.
You can tell what the internal sequence index is by examining the miniusf file as I suggested.
0x80053CF8 and such are addresses of code in RAM, this would only be useful if you had a way to set a breakpoint on a call to that address. This would be fairly involved and it wouldn't directly take you where you want to go, so I wouldn't worry about that.
I'll think about how to explain reading the hex more simply, could you be more specific about where you get stuck? I am going to probably be too patronizing in what follows, but I doubt I'll cover everything.
One thing that may be confusing is that numbers are stored in little endian, so the number 0x24050067 is stored as the bytes 67 00 05 24.
0x indicates hexadecimal (aka hex), but when I write out series of bytes like that it is also hexadecimal (so the 67 above is 0x67).
In the middle of a hex dump or a hex editor, the bytes of the file are just shown as hexadecimal numbers. A byte ranges from 0 to 255 (in base 10), so it is written in hexadecimal as 00 through FF; every byte is represented exactly with exactly two hexadecimal digits.
The symbols on the far right of a hex dump are the bytes of the file interpreted as ASCII; in general this is meaningless, but when there is text it serves as a good landmark. A "." is used whenever the byte doesn't represent a printable character.
The interaction of USFlib and miniUSF is this: The USFlib is loaded first (into empty memory, initialized to be all zeros), then the miniUSF is loaded over top of the USFlib in the same space. Both the USFlib and miniUSF are stored "sparsely", i.e. they specify only certain ranges of data. So anything not specified in the USFlib is assumed to be zero, and anything not specified in the miniUSF is assumed to be whatever was in the USFlib.
The SR64 (for Sparse Rom 64*) format used here encodes the sparse files as ranges of bytes. To reiterate an example above:
53 52 36 34 <- the ASCII codes for SR64, this starts the sparse encoding
04 00 00 00 <- the upcoming range is 4 bytes long
A4 B9 0B 00 <- and it starts at 0xBB9A4 in the destination memory space
67 00 05 24 <- and these are the 4 bytes
00 00 00 00 <- the upcoming range is 0 bytes long, this signals the end of the sparse encoding
----
(*) SR64 is something of a misnomer as there are 32-bit values everywhere, it came from N64.
edited 3:41 AM EDT May 3, 2014
At any rate, if I'm understanding this right, then the tracklist--at least for the USF set I have, which I got at Zophar, I think--goes like this (in general I use track titles from the rip, and OST titles are in quotes):
00 - "Toad Town" (plus variations)
01 - "Goomba Village Theme"
02 - "Battle Fanfare" (including first strike variations)
03 - "Hey You!"
04 - Jr. Troopa Battle
05 - "King of the Koopas"
06 - null?
07 - "Goomba King's Decree"
08 - "Attack of the Koopa Bros."
09 - "Trojan Bowser"
0A - "Chomp Attack"
0B - "Ghost Gulping"
0C - "Keeping Pace"
0D - "Go! Mario! Go!" (both versions)
0E - "Huffin' and Puffin'"
0F - "Freeze!"
10 - null?
11 - "March Ahead"
12 - "Fuzzys Stole My Shell"
13 - "Koopa Village Theme"
14 - "Koopa Bros. Fortress"
15 - "Mysterious Dry Dry Outpost"
16 - "Mt. Rugged Theme"
17 - "Dry Dry Desert Trek" (but no oasis variation?)
18 - "Dry Dry Ruins Quest"
19 - Tutankoopa's Threats 1
1A - "Forever Forest Theme"
1B - "Approach to the Mansion"
1C - "Boo's Mansion Theme"
1D - "Gusty Gulch Adventure"
1E - "Tubba Blubba's Castle"
1F - "The Castle Crumbles"
20 - "Shy Guys Toy Box"
21 - "All Aboard!"
22 - Ghost in the Toy Box
23 - null?
24 - "Jade Jungle Theme"
25 - Raphael the Raven's Tree
26 - "Welcome to Yoshi's Village" (both variations)
27 - "Search for the Fearsome 5"
28 - "Raphael the Raven"
29 - "Hot Times in Mt. Lavalava"
2A - "Escape from Mt. Lavalava"
2B - "Star Way" (plus ascent/descent variations)
2C - The Master Battle
2D - Beach Tunes
2E - Club Song (I think this is one of the radio station songs, though?)
2F - Golden Oldies
30 - "Clouds Over Flower Fields"
31 - "Flower Fields Rondo"
32 - Cloudy Climb
33 - Puff Puff Machine
34 - Sad Sun
35 - Happy Sun
36 - null?
37 - "Crystal Palace Crawl"
38 - "Cold Reception in Shiver City"
39 - "Detective Mario"
3A - "Snow Road"
3B - "Over Shiver Mountain"
3C - "Starborn Valley Trail"
3D - Madam Merlar
3E - Mail Call!
3F - "A Party at Peach's Castle"
40 - "A Kingdom in Chaos"
41 - "Mario's Theme"
42 - New Item
43 - null?
44 - Boo's Record Player
45 - Tutankoopa's Threats 2
46 - Kammy Koopa
47 - Jr. Troopa
48 - "Bill Blaster! Go Faster!"
49 - Petunia Under Attack
4A - "Shy Guy Riot"
4B - Underground Pipes (but no Shiver City tunnels variation?)
4C - Whale Meeting
4D - Forever Forest Warning?
4E - Fearsome 5 Found
4F - unused fanfare
50 - "Gates of Goomba Castle"
51 - Koopa Bros. Tricks
52 - "Koopa Bros. Keep Cool"
53 - null?
54 - Tutankoopa Appears
55 - "Tubba's Heart"
56 - "General Guy's March"
57 - "Lava Piranha Attack"
58 - "Huff N. Puff's Theme"
59 - Crystal King Appears
5A - Blooper Appears
5B - Mini-Boss Appears
5C - Monstar Appears
5D - Club 64
5E - unused opening
5F - "Beware! Bowser's Castle!"
60 - "A City in the Stars"
61 - "Shooting Star Summit"
62 - Journey to Bowser's Castle
63 - "Sanctuary!"
64 - "Siege on Bowser's Castle" (two variations, but I think there are supposed to be more?)
65 - Dark Dungeon of Bowser's Castle
66 - "Angry Bowser"
67 - "Bowser's Rage"
68 - "Wish of the Princess"
69 - Main Menu
6A - "Main Title"
6B - Bowser Takes Peach's Castle + Princess Peach's Castle (Bowser)
6C - Prologue
6D - "Princess Peach Saved!" (both parts)
6E - "Mario and Peach's Theme"
6F - Bowser Defeated
70 - "Twink's Theme" + "Goodbye, Twink!"
71 - Making a Cake
72 - Gourmet Guy's Ecstasy
73 - "Princess in Distress" (plus Peach's balcony theme)
74 - "Hang in There, Peach!"
75 - Stealthy Snoop
76 - Peach Caught!
77 - Trivia Game (includes intro, game itself, and ending)
78 - "Star Spirits' Request" (plus revival theme)
79 - Murder in Shiver City + Herringway's Theme
7A - Shiver City Mayor
7B - Growing Beanstalk
7C - Merluvlee + Merlee
7D - "Lakilester the Great!"
7E - Goomba Bros. Defeated + Goomba King's Castle
7F - Flower Fields Sunshine
80 - K64 Train Ride
81 - Whale Ride
82 - New Partner
83 - Dry Dry Ruins Appears
84 - Pleasant Path HP
85 - Mini-Game
86 - Moustafa
87 - Game Over
88 - Good Night
89 - Grateful Flowers (Petunia + Posie + Lily + Rosie)
8A - Entrance to Flower Fields
8B - Victory
8C - Popa Diva (long & short versions)
8D - Boos' Dance
8E - Level Up
8F - null?
90 - "Victory Parade"
91 - "The Celebration Continues"
92 - null?
93 - null?
94 - Mario Bros.' House
95 - "Story of the Stolen Spirits" (in three parts + Return to Peach's Castle)
96 - "Nice to Meet You" (JP version of New Partner theme)
So... did I do it right? =P
Anyway, from what I understand from the readme, as well as one or two past threads on this forum, the places designated as "track numbers" might better be understood as section numbers, since each contains four places for music (sparses), correct? I hear that most of these only repeat the same music over the four sparses for the track, but sometimes it'll unloop or something. But there are some instances (like 0x95) where there will be four different songs: the prologue in three segments, then the epilogue music. Some (like 0x02) will include variations like the three different ways a regular enemy battle can start (no first strike, your first strike, enemy's first strike).
One point where I'm confused is, where did you get the number of 143 for the track sequences from, hcs? I count only 141 main tracks--unless we include the ones I think are null, which comes out to about 150 or so. I'm wondering if I left something out? Or if the ones I've labeled as "null" above actually aren't?
Another question: If there are only four tracks per sequence, where did the nine or so Toad Town variations come from? How does that work?
Anyway, since there are four songs per track, I'd love to get my hands on the raw rip for this game hcs talked about in the thread linked above, which contains 608 tracks (even if most are duplicates).
Perhaps in that raw rip one might find the missing tracks (discussed here, particularly Benjamin's post on the last page), songs such as the Dry Dry Desert oasis, Shiver City variation of Toad Town Tunnels, as well as other tracks missing an intro?
(Heh, it's amazing how complicated the music for this game is. I think next, I'll go for something simpler, like Super Smash Bros. or Super Mario 64. =P)
edited 1:42 PM EDT May 4, 2014
edited 1:44 PM EDT May 4, 2014
However, despite how easy it was with that last one, I can't seem to locate the track numbers for any of the Majora's Mask tracks. Opening up a miniusf in WindHex gives me massively more data than any miniusf in the other 5 games I've been looking into. I've tried looking at byte references early in the data, then down toward the end, but no luck. Does anyone have any idea how to locate track numbers/song select values for this game? Thanks!
(Also, if anyone still has that raw original 600+ USF rip for Paper Mario, I would be much obliged.)
- the track index
- some ROM offsets
- the whole sequence
- the sequence and some bits of loaded samples
- after the game has synthesized a few frames with all kinds of partial state scattered everywhere.
It sounds like the Majora's Mask rip is more like the latter.
Good news, though, MM's music mod is well-explored through gameshark codes. Here is a simple list:
Majora's Mask: Music Values
If you want to hear these in-game there seem to be a few different actual GS codes to use, I don't know which of these will work with which versions of the ROM.
1
801FD3AD
(credited to DarkSerge)
2
801F77DF
also 801F484D - "Ocarina Music Modifier"
(credited to Krijy, PAL version?)
3 (youtube)
801D6FB8 - This changes the instrument patch used when Link plays his instrument
(from ps3punk7890)
edited 10:55 PM EDT May 17, 2014
Thanks for the link (well, all of them) to the MM music values. The nomenclature on those sites was so vague (and a few times even outright wrong) that they were almost unusable as a source. However, this proved to be a blessing in disguise, as I took the ones that were obviously correct and looked up their hex values in WindHex. From there, I determined how to find track numbers in MM's USF files:
The trick is by using hex value 80. For most songs, between offsets 100 and 300, 80 will be the next byte value after the track number byte (this is even more obvious when you realize the track number will only be at the beginning of a string of four bytes). So for instance, "Astral Observatory" is track 3A. Which means, somewhere after the 200 offset (I'm just going on memory here), you'll see 3A 80 00 00 (or that last might be 04?). Plus, on the next line or two, you'll see the track number repeated, generally followed by 00 values. That was the trick to cracking track numbers for MM. (It's a little bit different for the Ocarina songs as you won't find the track value until after the 600 offset with 80 next to it, then again with 00s after the 700 offset.)
So this was a very good learning experience for me! Next I'd like to try my hand at determining tracklists for Super NES and even NES games.