Zelda Twilight Princess .bms Music Rips by TurbomanAlco at 6:11 PM EDT on June 2, 2012
After reading some threads on the forums I found that it is possible to rip music from Zelda Twilight Princess by extracting .bms files and converting them to midi files using this php source code. Then you can use the instrument banks from the extracted .aw wave data to plug into the midi file. This is where the problem comes in. I need a way to
1. Determine which .aw file or files the .bms file uses
2. Determine which extracted .wav files from the .aw files the .bms file use for each instrument or track.
I don't realy have any knowledge on hex editing and analyzing so any help would be appreciated. I also heard about Valiant's TP music rips but the links are outdated and the qaulity may not be as good as direct rips.
I have MIDI files converted from Super Mario Galaxy BMS files. They were sent by furry bob. I'd also like to know how to do the same as what the above user wants to know, but with SMG. But then, if you knew how to do it, you'd have done it anyway, right? There are 53 files I have... funnily, there are 53 tracks on the 2nd disc of the original soundtrack.
Yeah, I have some 'aw' files from SMG... I also managed to get wav files of the Grand Star and (Normal) Star jingles, as well as the short drum loop that plays when you're holding a shell underwater. If I were to follow how it's done with TP, I should be able to do a similar thing with SMG, right? They both use aw files, and bms, but... inb4theyhavedifferentstructuresorsomethingblahblah
Yeah, the structure may be slightly different in some ways but it should overall be similair. You might just need to do some extra research and you'll be fine.
You go into the file system of the game, go to the audiores folder, then go into the sequences folder, extract the sound seqs arc file. Then you use the rarc dump tool to extract the bms files from the arc file. Then use the bms to midi source php code that I mentioned earlier to get the midi files.
So .... has anyone found any way of actually playing sequenced audio, as it originally should sound, and not through MIDI? As in, has anyone been able to say, get this sort of thing?
http://www.youtube.com/watch?v=vcT6_zVHb6Y
This, as you may or may not know, was ripped by furry bob using some hax skilz.
But is there any way whatsoever (that's been discovered) of playing sequenced audio from Wii or GameCube games, such as Super Mario Galaxy or Mario Kart Wii?
Not possible at all the way you're talking about. None of the music is actualy a recording of live music. The only way to do it through actualy ripping (and not recording the music directly through the game) is to get a midi file from the bms files and use the music banks from the aw files and put them together. It can't be any simpler than this unless some one comes out with a method of finding out which music banks go to which bms file or until there's a tool that autmaticly does it.
No. QSF is for the Capcom Q Sound something or other. It's an arcade unit (see CPS2 on Wikipedia). I already mentioned QSF here. Personally, for NGC/Wii xSF, I would suggest either RSF (taken already...), or WSF (I think that's still available). Because the Wii and NGC are so similar, the xSF implementation of either sysetme would, likely, be very nearly identical. Thus NGC/Wii should be able to be conglomerated under one extension.
Just FYI, Neill Corlett created the QSF format (before there was even a NGC emulator of any kind, I think. I could be wrong), and most of the rips, before he "disappeared." As such, it's been around for years, but no one has ever done much with it. Mouser X over and out.
Considering that 2SF is for the NDS, wouldn't it make a lot more sense for 3SF to be for the 3DS? 2SF-NDS, then they put out the next NDS, which is a 3DS.... 3SF-3DS. How does 3SF for the NGC make sense? Wii SF (WSF), or Cube SF (CSF) sound fine to me, but where does 3SF come from (in relation to the Wii/NGC)?
Obviously, this entire conversation is completely irrelevant. Whoever actually does the work to create the format (aka, makes the initial rips, and a player for said rips) is the one who will make the name. If they decide to call it *.!@&, they can do that, because they're the one who did the work. Hopefully whoever actually does create it, doesn't call it that just to spite me.... Mouser X over and out.
The (possibly not full) list of Portable Sound Formats is: PSF or PSF1 = PlayStation Sound Format PSF2 = PlayStation 2 Sound Format SSF = Sega Saturn Sound Format DSF = Dreamcast Sound Format USF = Nintendo Ultra 64 Sound Format QSF = Capcom Q-Sound Format GSF = Game Boy Advance Sound Format 2SF = Nintendo DS Sound Format 3SF = GameCube Sound Format
The 3DS sound format for sequenced audio could be something like.... CSF, as in "CTR", as was suggested.
However, I think most 3DS audio is streamed. So it's likely to be something like BRSTM, ADX, etc.
What do you mean it works? Did you actualy some how find a way to compile it? All you have to do is get some webspace on the net and drop the folders into it using an ftp editor. Then you go on the site using a web browser. There is no compiling unless what you mean is you did that.
Look up at this sites URL. See the forum.php part? Thats because on the webspace is a file that is sort of like an advanced form of HTML that has more codes and all to make all the nice little gadgets and all for this page. The php source is just a copy of the original php code from some website that allowed you to convert bms files to midi files. The site is now gone so you can only use the php code to put it on your own webspace to convert bms files to midi files. Just simply drop the file from the php source into your webspace via ftp and your good.
I think the main issue is figuring out which samples to use. I got indoor.bms to sound decent using FL studio and assuming which .wavs to use, here's a link if you'd like to hear how it came out:
Did you modify it to seperate them by the aw file they come from or the bms files they correspond with? Does it use any of this information to determine which folder to dump the waves?
That's the data dumped from the baa file I believe and tells you the file system of the music banks. Too bad the wyster dosn't extract information like that otherwise it would be much easier. I thought you may have used that information to make the modified wyster. How were you able to determine which aw file goes with the proper bms file so easily? They're all mixed up for the most part.
Also in that dumped data, SE stands for sound effects which make up most of the wave banks and BGM is the location of the actual instruments used in the bms files organized by the bms file they're used by.
Yeah, I figured out the BGM and SE stuff myself, and to decide which samples to use I just listened to all the ones until I found a set that sounded like they belonged in the indoor track, using the actual game as a reference.
If someone who is better than I am at C can modify wsyster to use the BAA files, then we'd be getting somewhere. Is there a specific tool used to dump the BAA, with source? Because that could possibly be merged with wsyster.
I am not a C programer unfortunately. And the tool to dump the BAA is that baad program by hcs. I don't know what the bst program does. Baad just extracts the music bank information and dumps it as bnk files which then tells the wyster what to do for when extracting the waves. So somewhere in the baa file I bet you'll find the connection between the bms files and waves, and maybe enough to develope a new program to automaticly give us what we need. I believe hcs got that dumped text file by extracting hexadecimal data from the baa file. Anyone good with interpreting hex?
You use rarcdump from this site. There's some other good stuff there too. The tool will make a folder of the same name as the arc folder where it is extracted. In there should be all the bms files. If it's the midi files you want, you need to wait for a tool to be developed to extract bms files directly on your pc. Otherwise you'll have to host the php source I mentioned earlier to extract the bms files and if you have free hosting you can't extract all of them (the tool uses to much processing for larger bms files and free hosting dosn't allow past its limit). That's yet another thing that needs to be done to make the best achievable rips of Zelda TP/Super Mario Sunshine music and I unfortunately don't have the knowledge to do so.
UF's adventures in BMS Unknownfile said that sequence.arc was not an RARC file... He wrote a program to extract the .bms files from sequence.arc, but I don't know how to get the program.
I gave you the link to rarcdump which if you look on that site you'll see it says it's a tool for extracting arc files. The rarc part is just the name of the program. You don't need a program for specificly extracting only bms files from the sequence.arc. The rarcdump tool will just give you everything inside the arc file including non bms files. It's very simple to use. All you do is make sure the arc file is in the same directory as rarc and drag and drop it into it. Then a new folder will appear in the same directory with the same name as the arc file you extracted. In this case it would be a folder named sequences.arc and inside will be the bms files.
Oh yeah, it's been awhile since the last time I used the tool. I just did a test to see what would work and I see I missed a step. You drop the arc file into yaz0dec.exe and you get a rarc file. Then you take that and put into rarcdump and you get the output folder with all the contents in it. Sorry, I gave misleading information. And good work maxton. That site is what the php source I've been talking about looks like when properly hosted. That should allow us to easily get bms files without any problems. Now that we're all on the same page (I hope), all we need is a way to know what insturments go with which bms file. That is unless anyone needs an explanation on how to get the insturments?
I think furrybob disabled the SFX and actually recorded Super Mario Sunshine. Eh, He made some random Yoshi variations of stuff, so maybe he was able to extract them.
Um, Dutchie? Are you thinkijg of Super Mario Galaxy's bms sequences? Furry bob never extracted any bms files from Super Mario Sunshine. There's no OST either.
Well maybe that's the difference. If the sequences.arc of sunshine/galaxy or whatever don't work with yaz0dec then that must mean they are compiled differently. This is not anything new however. I have seen some bms from them before thus proving it possible. It's just a matter of finding the right tool. Nulling out the SFX is one way to go but I think we should try more high qualitly rips from midi files and the BGM banks.
Yes they may be fine but there still may be some flaws such as missed songs, SFX that wasn't completely nulled, and some loss of quality. Not to mention that it takes forever to go back and re-beat the game and record every song. Also nulling out the SFX can be a realy big pain. I believe the safest and best achievable quality is through direct rips. And believe me, it will take alot less time and struggle.
Geting the bms from the sequences.arc isn't anything new and neither is getting the bms files to midi files. I'm sure someone made a tool just for this type of work. Unfortunately gamecube ripping isn't quite the popular subject among the internet so finding the right tools is very difficult. Luckily, I found the tools I need Zelda TP ripping. It's just a matter of knowing what waves goes with which bms file.
unfortunately i have been trying to do the same thing and even through my years of experience i have not come out of an easier way than that. i do he that a program will be made possible soon so that projects like that will come easier for us. still waiting on any furture updates on this.
Nearly the half is finished. I created manually a table for each instrument which contains the sample ID, the number of the AW file, loop start and loop end sample positions, key range, root key and the name of its .wav file that belongs to the AW. I wrote a tool to simplify the process of getting these information from the sound banks but unfortunately I can't find it anymore ):
The track names of the converted MIDI files indicate which instruments are used (but there are a few sequences which aren't converted properly). Some tracks also contain instrument changes like in field_link_day.bms.
The numbers on the instruments in the midi files don't make any sense at all. How exactly does this work out? According to itsacatfish, the .bms file should indicate the wave thats used. Im not too good analyzing hex code but I can tell you they didn't write it out in plain english or numbers but in some gibberish string. Probably just information we all know already but thats all I can make out of the hex code. How exactly are you doing this Yuuto?
The game has 240 instruments in total that are actually used in the sequences. They are defined in Z2Sound.baa.1_5.bnk (extracted from Z2Sound.baa using baad.exe, I don't know if the other banks are relevant). The track names of the MIDI files show the number of the instrument used. The .bnk contains information about all the samples the instruments use (sample ID) and the key range. Z2Sound.baa.1.wsys contains information about each BGM sample, namely root key, length, loop positions and the associated AW file. I think the only way to find out which samples a sequence uses is to look at its instruments.
I could explain the process of creating a soundfont out of this in detail, that means which hex values are neccessary and so on, if you want.
Here are some examples of soundfonts I've finished (I grouped the instruments just for testing purpose). Try to apply them to filone_forest.bms and deathmountain01.bms. Faron Woods Death Mountain
Yes please do explain this in detail. But what exactly are the soundfonts? Are they an alternative to midi files but maybe more advanced? And that would seem true that there are only 240 instruments because there are 2,453 instrument waves in total and it does seem like there are alot of repeats. Thats definately a big relief that it could be alot easier to work with.
A soundfont is like an instrument from TP's sound bank (better: a collection of instruments/presets), it contains samples for the playback of sequences (in this case for MIDI files). Viena is a free soundfont editor, you can use it to 'play' the soundfonts, too. There's another program called SynthFont for the playback of MIDI files, there you can actually use the soundfonts. Both you can find at www.synthfont.com
This seems realy cool. I'm looking forward to this. So when the soundfont is finished it should be rigged up to work in synthfont with any midi file right (with an exception to the ones with mixed up instruments)?
Right. At the moment I create a soundfont for every instrument, which is much simpler. Later I'll assemble all instruments in a single soundfont. Another cool thing is that the MIDI files can be rendered using SynthFont and the soundfonts. Due to the limitation to 16 tracks per BMS some sequences use instrument changes, which aren't included by the PHP script. For the moment, these can be done manually in SynthFont.
But there are still some other problems with the sequences, e.g. the missing reverb values and errors during conversion. See the wrong notes in e_dekutoad.bms.mid and e_goma01.bms.mid. Maybe I will try to write a new conversion program later.
I imagine you'll hit quite a few problems like that. I'm sure the end result won't sound quite the same as the game, somehow, indeed due to things like DSP, possibly bi-directional looppoints which soundfont doesn't support, non-standard controllers, maybe the samples themselves would sound different just because it's a different engine using them... like interpolation and stuff like that. I'll be interested to see how it turns out nonetheless.
Apart from the missing reverb, the improvised envelope data (attack, hold, decay, release) and the slight errors during conversion it currently sounds almost like in the game. Even vibrato is supported. Btw what are bi-directional loop points?
I don't see much of a better way to do it. Dosn't dolphin have a bunch of dsp problems itself? Besides reverb isn't hard to replicate in other programs like audacity and fl studio. The looping isn't anything to hard to do because all of the songs I converted to midi files are already looped and the looping points are the begining and end meaning the whole song is looped. I wouldn't be worried about any of thoose things. Also they use a special technique in the sequences that give a reverb like effect probably meaning that there realy wasn't ever supposed to be a real reverb. By that I mean they double some of the tracks, move the notes over a bit, tone down the volume, and wala. You have a fake reverb like effect. Try it or look at some of the converted .bms files.
The sequences have a reverb command in some tracks. I took some MIDIs and rendered every single track. The tracks with a reverb command I edited manually in Audacity, and it sounded like in the game.
Okay, now the process of retrieving the instrument data. I think this could be a bit difficult to read and confusing, so maybe I'll upload a format specification later.
As I have already said you will need Z2Sound.baa.1.wsys and Z2Sound.baa.1_5.bnk. The .wsys file contains sample information, the .bnk instrument data, so at first open the instrument bank. The file is structured by chunks, each chunk contains a different kind of information and starts with a 4-byte (DWORD) identifier in ASCII format (FourCC), so it's easy to find. At the very end of the file you will find the "LIST" chunk. The DWORD after that is the chunk size, then follows the number of instruments. Note that all values are DWORDs and in big endian format.
The number of instruments is followed by an offset for each one (it starts with the offset for instrument 0). At this offset you will usually find the FourCC "Inst". Jump 4 DWORDs forth. This is the number of samples the instrument uses and is followed by a 24 byte sequence for each sample. The first byte is the upper key range limit, i.e. when this is 46 (2e) in the first sample the key range is 0-46. Jump 12 bytes forth. This DWORD is the ID of the sample. All values I didn't mention are currently unknown.
If the FourCC is "Perc" it is one of the two percussion 'instruments'. The structure is similar to the LIST chunk. The FourCC is followed by the number of percussion samples followed by an offset for each sample. The index of the offset indicates the key the sample uses (empty offsets indicate that the key is empty). This will lead to the "Pmap" FourCC. The byte 12 bytes from the beginning of the FourCC is the pan value (00 = fully left, 7f = fully right). The DWORD 28 bytes from the beginning is the sample ID.
The instrument bank does also contain envelope data but i haven't figured it out yet, so the values for attack, hold, decay and release must be done manually.
That was everything about the instrument bank so far, now to the sample banks. When opened the .wsys file you should set the bytes per line to 56, this makes reading the values easier.
At first search for the "WINF" chunk. Then search the sample you want by its ID (search for a 4 digit value, i.e. 1f0 -> 01f0). I know this method is not very precisely. When found, the byte in front of the ID is the number of the AW file. Then count the lines down to where the AW number is (start at 0, count how often "ÌÌÌÌÿÿÿÿ" stands in front of it).
Then change the bytes per line value to 44 and search for Z2BgmWave_X.aw where X is the AW number. Look at the table above that string and go in fron of the first "Ì" in the top line. Then go down the number of lines you counted before. Go two bytes forth. This is the root key. After that byte go 14 bytes forth. If this DWORD is 0xffffffff the sample loops and the loop start and loop end DWORDs follow. If 0, it does not loop. then convert the number you've counted into hex format and this this the name of the wave file.
I know this is very complicated and inaccurate, but this worked fine for me. Maybe I'm going to write a tool that extracts that values automatically.
Hmmm, I don't understand the .bnk part to well but I was able to get some results on how to get the instrument. I'm looking for instrument 162 so after I was at the WINF chunk I did a search going down for 00A2 or 162 in hexadecimal. Then I looked at the binary in front of that and it was 2D or 45 in decimal. So then a did a full search for Z2BgmWave_45.aw and I did the counting the lines thing and I got Z2BgmWave_45.aw_00000002 which dosn't sound the least bit right in the twilight.bms.midi on the first track. Am I doing this right? And I can't apply the instrument data you retrieved from your tool to anything. How does that all work?
You need tolookup the ID of the sample instrument 162 uses. That would be the .bnk part, which you now can leave out because I uploaded ibnk.html. There is every instrument with its sample IDs listed. Instrument 162 has only one sample, that with the ID 194. Try it again with this ID.
Ahh, this time it worked but my counting down was off by one number because I started with zero as a pose to one. Now to apply this to a soundfont if i can figure that out because it dosn't sound like it's at the right pitch in FL Studio indicating that it dosn't have the proper root key that I need.
Thanks for mentioning that. Otherwise I probably would have went with 66 which is for Z2BgmWave_14.aw_00000000C and I need D but I messed it up when I counted one off. I'll probably need that number anyway cause that whole aw file seems to have most of what I need for the Twilight theme. I'm starting to like this soundfont tool. I tried it out and it's perfrect so far. Now for the next few instruments. Come to think of it, you could probably do this all from FL Studio using the Direct Wave plugin which works practically the same way if anyone wants to try that.
I've run into a small problem. I see in your soundfonts that there is proper looping. Was that done manualy or by values given by either the .bnk files or the .wsys files?
It's just when the sample plays backwards when it hits the loop point, then hits another loop point and plays forwards, etc. It most likely isn't used in TP, but it has some advantages... makes looping easier at least.
Hopefully the vibrato rate/depth is the same as in the game, also.
The vibrato is a bit weaker than actually in the game but I think this can be fixed in the conversion script.
by TurbomanAlco at 11:07 AM EDT on August 10, 2012
Oh, so it's right after the loop part. That should make things a bit easier. There's vibrato in this music? I never realy noticed mostly because I thought all the songs were prerecorded until I found out.
That's good. But there's something I don't really understand: When I looked at instrumen 23 and searched for the samples 2f9 and 2fa I could not find them, the same thing with some others, too.
by TurbomanAlco at 12:11 PM EDT on August 10, 2012
What .bms are they from? They might use just a single wave without anything special or somthing. If you have to, go here and they got a bunch of cheaply recorded Twilight Princess songs and you can play it by ear. They should have every song just about. This probably the worst way to have to go but some parts may be recognizable and can be linked to a wave.
Instrument 23 is used in 1v1.bms.mid (and the other to versions). As far as I know this should be a pan flute. In this sequences they also use #22 which is an instrument from the Goron Mines and that can't be right.
by TurbomanAlco at 12:44 PM EDT on August 10, 2012
Maybe they reused an instrument. I got a strange problem with instrument 120. I know for sure that it's Z2BgmWave_14.aw_00000003 because it's the last instrument in aw 14 and it sounds good in the sequence. The problem is that it's slightly off and it overlaps it's own notes in the sequence. You would have to hear it to know what I mean. Also I had to geuss because on that instrument it says percusion but when I went to the percussion page there is no instrument 120. Is there a special way to do this?
Actualy come to think of it I know of two waves that sound exactly like eachother that fit in for instrument 120. I'll try the other one.
Percussion 0 is instrument 0, percussion 1 is instrument 120. The first value is the key, the second the pan, the third is unknown and the last one is the sample ID.
Note that this 'twilight' sound on key 60 has the root key 59. I don't know if that is indicated anywhere, but with root key 60 it sounds a bit to low.
You simply have to set the keyrange to the value on the left in the percussion table, in this case 60 (in Viena 60-60). Then you set the pan to the next value (in Viena this goes from -100 to 100).
I don't understand what the problem is.
I uploaded Instrument 120 so you can see how I did this.
Thanks, your probably right. It's probably on a pitch that's to low causing it not to fit in thew whole note. I'll try your soundfont. This is the last instrument needed to finish this .bms.
The problem is that if you listen to instrument 160, the note appears to be to short for the wave or somthing. The point is it sounds un-natuaral. You have the same problem?
You're right, that sounds weird. It seems that the next note comes to early. I compared it with a recorded version of the Twilight Theme. If I increase the root key it is slower, like in the recorded version, but then it's too low. I don't really know what's wrong with that.
EDIT: I solved it. You have to set "Tune, Fine" to 25.
Since I don't have experience in PHP (I'm glad that I understand it to some extent) I'm going to rewrite a converter in C#. Using the PHP script as a base, of course.
That would probably be better anyway, keeping it as an online script would become a POTA at some point, and having a local executable would allow for a more intricate program, possibly including not only MIDI conversion but also incorporating sample extraction
Okay a first version is finished. The "length of tempo" issue from the PHP is fixed, but is doesn't loop (I mean the notes aren't put twice in the MIDI, which keeps the files smaller). It works well for the normal sequences, but not really for the adaptive/dynamic ones yet. In the main header of these there appear some strange 0x00 markers, does anyone have an idea what that could be?
Cool, it would be very useful to have an executable version of the bms converter. Unfortunately that's all realy out of my knowledge. And it appears the fine tune fixes the song a bit. I geuss you notice it more with the higher quality.
by TurbomanAlco at 12:05 AM EDT on August 25, 2012
Ok, I finished the Queen Rutela's theme but to completely finish it I'll have to put in the reverb because otherwise it dosn't seem completely perfect. I assume that the B8 02 hex values in the bms code are in order by the same way they are listed in the midi output. But the thing I dont get is how to use the number following it. Apparently that reverb value can't go over 127 or 7F in hex. But what does that number realy mean? Is it the roomsize, dampening, dry level, or reverb time? There's lots of values when working with reverb, not just one.
It is the dampening. When I did this in Audacity I used 0,16 sec for delay (propably always the same) and the value for dampening, where 7f would be 0,5).
I think I've found the reason for the wrong instruments in some of the seqs. The instrument change command (e3) is always preceded by an e2 command which mostly has the value 0b. The wrong instruments use another value, so far I've seen 0d and 33 (in the howl songs). Perhaps this is some kind of instrument bank change?