Anyway, I've been doing some research on the game these past few days. Thanks to PCSX2, MHS (Memory Hacking Software) and a hex editor, I'm starting to get fairly confident that SOUND.MGF may just be sound effects and some... other data, which at the most controls how/where the sequenced music plays. So it looks like there's really nothing of much interest in it (unless you do want the sound effects).
The top chunk consists of S-FORCE (.sfc) files, which is a 3D sound format. I noticed that when I nulled the "forest.sfc" file (0x49F800 in SOUND.MGF), the growls that usually appear when you pass through the forest are no longer there. Everything else seems intact.
The bottom chunk (0xFC6000) contains .sdb files. These, as kind of previously mentioned, seem to just control where/how music plays, but I have no idea how they really work. I guess they could be sound banks of some kind...
So even if someone did extract the data of SOUND.MGF, you'd still have to figure out a way to play/convert the stuff... And I don't think anyone knows anything about it. In fact, this is the first time I've heard of S-FORCE. I only found two games that apparently utilize it: Silent Hill 2 and Kingdom Hearts II. I also found a press release regarding S-FORCE for whoever cares.
If someone thinks differently (or even knows) about the SOUND.MGF file though, by all means prove me wrong. I'm not saying I have to be right, but it is an educated guess.
Also, the reason I'm calling chunks "files" is because the game actually does have a virtual file system kinda thing. You can see all the paths in the main executable (SLUS_202.28). Here's a structure and an example...
Table 1:
ulong address of sub-file attributes (pointer to main file or larger chunk of it, offset, size) (->table 2)
ulong address of sub-file path (->table 3)
Table 2:
Each entry has one of three initial values before it's loaded into memory. So, depending on that value...
ulong 0x03 00 00 00 -- physical file
ulong address of filepath (->table 3)
ulong offset (sector) of physical main file
ulong size (bytes)
ulong 0x23 00 00 00 -- virtual file
ulong same as for physical files
ulong same as for physical files
ulong same as for physical files
ulong 0x50 00 00 00 -- virtual sub-file/chunk
ulong address of entry of main file or a larger chunk of it (->table 2)
ulong offset (bytes) in the file/chunk
ulong size (bytes) of sub-file/chunk
Table 3 contains null-delimited filepaths only.
It probably seems a bit confusing, so here's a demonstration - let's find out in what file forest.sfc is stored, as well as its offset/size:
Prerequisites: Silent Hill 2 (the disc itself or an image file), PCSX2 and MHS.
First off, I feel I need to make it clear that when the game's main executable is loaded into memory, a lot of values are changed, stuff is loaded above it, etc. So the addresses in the static executable doesn't really make much sense unless the game is loaded. Also, you need to know that the address in memory at which the actual game data begins, is going to be 0x20 00 00 00 and not 0x0. This is because MHS reads physical memory, and not exclusive PS2 memory. So you need to account for that, where needed, by adding 0x20000000 to addresses that are stored in memory by the game and subtracting from addresses that aren't stored in memory by the game. Lastly, keep in mind that all values are big endian, so you'll also likely need to reverse the byte order when using functions such as goto (Ctrl+G).
Start by fully loading Silent Hill 2 in PCSX2. Then open the process for PCSX2 in MHS. Now make a string search for forest.sfc (make sure "aligned" and "match whole word only" are NOT checked). Open the first address in the results with the built-in hex editor. Note the address of the start offset of the path (if you're unable to see the whole path, use goto (Ctrl+G) and enter an appropriate negative decimal number to backtrack from current position, as you won't be able to simply scroll up unfortunately.) Go back to the main MHS window and make a pointer search (exact value; make sure "find only static pointers" is NOT checked).
If you get multiple results, open the first dynamic (non-green) address in the hex editor. You should be somewhere in table 1 now. The 4 bytes to the left of the found value should be the pointer for the file's entry in table 2 (where all the good stuff's at). Use goto to get to the address.
Now you should be somewhere in table 2. For this particular case - forest.sfc - we can see that it's a sub-file/chunk entry (0x50 00 00 00). Note the offset and size values of this entry for now, as those are the attributes for forest.sfc. Next, let's go to the address being pointed to. This is another sub-file/chunk entry, which simply defines a larger chunk at the top of a file. Again, note the offset/size values. Let's again proceed to the address being pointed to. We finally end up at an entry for a physical file. Let's see where the pointer value takes us. That should have landed you at the filepath of SOUND.MGF!
So, going by the info in table 2 and 3:
- forest.sfc has offset 0x49F800 and is 0x08F63A bytes in size
- the offset is relative to a chunk defined from 0x0 to 0xFB6000 (i.e. the offset is still 0x49F800, duuh)
- in file "SOUND.MGF"
In short: 0x49F800 to 0x52EE39 in SOUND.MGF is forest.sfc
Here is the full filepath table in a human readable format: https://dl.dropboxusercontent.com/u/48454461/misc/fparr.txt
Now, if only someone could be awesome enough to figure out how to make PSF2s with those module files, I'd be forever grateful.
edited 2:05 AM EDT April 12, 2014
edited 11:09 AM EDT April 12, 2014
psscut.exe will extract all individual PSS files from MOVIE.MGF, and they go in the same order as the file system says:
data/movie/deai.pss
data/movie/end_dog.pss
data/movie/end_mria.pss
data/movie/end_rbrt.pss
data/movie/end_scde.pss
data/movie/ending.pss
data/movie/flash.pss
data/movie/gero.pss
data/movie/hakaba.pss
data/movie/hei.pss
data/movie/knife.pss
data/movie/korosu_a.pss
data/movie/korosu_b.pss
data/movie/murder.pss
data/movie/open_bgm.pss
data/movie/open_voc.pss
data/movie/rouya.pss
data/movie/saikai.pss
data/movie/toilet.pss
data/movie/water.pss
The problem is there are 21 PSS files inside MOVIE.MGF, while filesystem gives names only to 20. Weird.
The files are extracted (via psscut) with names 000.pss~020.pss.
The last file 020.pss is the last name on the list = water.pss
018.pss is toilet.pss, so the missing name is for 019.pss.
I'm tempted to just call it "memory.pss" and be done with it. The cutscene shows a black & white recording of Mary by James, so I think the name would be appropriate.
Btw this thing is the same for Japanese version as well (SLPM_650.51), I checked - only 20 names and 21 files.
edited 4:18 PM EDT April 13, 2014
EDIT: Well, it does seem like it is something unused. But someone's gonna have to verify that, because as I mentioned, I haven't played through it yet (but even then, it looks like there's a lot of different scenes that you may or may not see depending on... stuff, so...).
edited 12:38 AM EDT April 14, 2014
Also it seems unlikely that one name could be given to two cutscenes at once. For example there's korosu_a.pss and korosu_b.pss - so they do separate two cutscenes into A & B parts even if those cutscenes of similar context.
I loaded Iso in PCSX2, created a save state, and looked at it in Hex editor, it's the same thing - the last file names are toilet.pss & water.pss, even though there's clearly a PSS file between them in MOVIE.MGF. Strange indeed. Even the save state doesn't have a name for it.
I guess the only way the game would know what file to play when it needed to play that nameless video file is by its position. Unless there's some ultimate true table of contents hidden somewhere that we haven't seen yet.
And just to reiterate I'm fairly certain that all other names are appropriate for other files from top to bottom. If translating japanese names (e.g. hakaba=graveyard, rouya=jail, korosu=to kill) it all makes sense according to what is shown in those cutscenes.
edited 1:33 AM EDT April 14, 2014
edited 1:35 AM EDT April 14, 2014
Does anyone know if there's a place with PS2 save states for SH2 (or in-game saves)? What part in the game would we be looking for (like.. area name)?
edited 9:28 AM EDT April 14, 2014
The unnamed file 019.pss between toilet.pss and water.pss is actually a first part of murder.pss. It's exactly the same frame-for-frame first 36 seconds of a flashback James is having in which it is revealed that he kills Mary.
I must've didn't realize it before but there's no instance in the game where he can have that flashback without remembering the whole thing - killing his wife. Which is exactly what 019.pss is - it doesn't show him killing her. So for what ever reason it was included in the game but was left unused. Makes you wonder if there was suppose to be a path where James doesn't even become aware of his crime.
Also I checked with PC version of the game, it doesn't even have that short version of the video, only the whole thing: murder.bik.
PC version uses bik codec for their videos and needless to say it is heavily compressed, much more than PS2 version that is. PC videos are not hidden in any container and are just in a "movie" folder and add up to ~600mb while PS2's MOVIE.MGF is 1.58 GB. PS2 has the best quality of SH2 cutscenes to date which is why I was after them with proper names, but somehow didn't think of matching the names to PC version.
On that note Silent Hill HD Collection would be a prefect opportunity to release a true-HD renderings of those cutscenes but they turned out to be a simple upscale job of bik videos from PC. A very lame and lazy way of doing things. Just like everything about HD Collection was. They probably didn't even bother to ask for original hi-def files stored on someone's harddrive at Konami, or Takayoshi Sato - the main guy behind CGI is Silent Hill 1 & 2.
edited 3:10 PM EDT April 14, 2014
You mind linking to file 019? I know there are a few unused images including test images I found in the HD Collection.
https://mega.co.nz/#!o01RUTqZ!qlC1fIGszyM4eMuk1aHu9I1rluJUBr-1G8Wm_dNYFmg
Also the reason why there are unused images in HD Collection is because the source code of games (SH2 & 3) that Konami had on file and gave to makers of HD Collection turned out to be earlier builds of those games, and had bugs not present in retail versions.
edited 4:18 PM EDT April 14, 2014
So has anyone figured out where the tracks from the Born from a Wish ending ("Terror In The Depths Of The Fog" and "Love Psalm") are hiding? I'm really curious about Terror in the Depths of the Fog especially, because it doesn't seem like it plays fully anywhere in the game (or does it?) It's my favorite track from SH2, I think. I know it's not in SOUND.DAT, and obviously it's not in MOVIE.MGF either. I doubt it's in SOUND.MGF... And seriously; no one here has actually managed to convert/play back anything in the SOUND.MGF file, right.. ?
edited 5:07 PM EDT April 14, 2014
Since that track was only used in a real-time cutscene (AFAIK) it is likely that there's only a short version of it in game's files. Which is not an uncommon practice I'm afraid.
As far as I can tell, there were no Greatest Hits nor Director's Cut PS2 release made 2001 (when the OST was released). Or was there? If not, you sure it's not on the Japanese release? I'd guess it'd be a Svag (Sony VAG) at the bottom of the SOUND.DAT file somewhere (the top half are sequenced tracks), probably with the dialogue and everything. The original US release has 109 "dialogue" tracks in SOUND.DAT.
edited 9:53 PM EDT April 14, 2014
So as it stands I don't think it'd be any different on console versions - track is gonna be a part of a dialog audio file used in cutscene.
As for the time it was released, well here's a breakdown:
The first versions of Silent Hill 2 (without Born from a Wish) came out in Sep 2001.
Silent Hill 2 OST came out in Oct 2001. (it had music from Born from a Wish)
The first Director's Cut that featured a sub-scenario "Born from a Wish" was released Dec 2001 on Xbox in North America.
So it's not all so far apart. Technically yes the OST precedes the appearance of the track outside of game. So a person who happen to own the 1st print of the game and the OST next month had all rights to go bananas and question why that track is on the OST but not in the game.
But the contents of Born from the Wish could be well in development by the time the main game went to retail stores or shortly after. All the music used in that sub-plot could be finished way before the OST was released.
Anyway the initial Jap PS2 version doesn't have it (SLPM-65051), the one that would have it is SLPM-65098 and other re-issues. But I'm pretty sure it would be just a dialog track in SOUND.DAT.
Here's nageru.adx.rar from PC port:
https://mega.co.nz/#!xtF3mJbY!bpKqVk5lU5qAhpBW0_wIwtawBePQTS8p8I_GOKGMYrQ
edited 11:31 PM EDT April 14, 2014
I know a few are in the retail version. I think they are in both HD collection and retail. I just only remember seeing some in the HD collection.
Have you also found Love Psalm? What's the filename/location for that one? ;)
edited 9:52 AM EDT April 15, 2014
https://mega.co.nz/#!wtc2iA7S!C3_XAwR1_QXXCAwkCTjNaE9NsbwegnrX8ZvFyFanzrY
The MEGA link gives a "temporary error" for me currently. Maybe it'll work later. But ah.. I was mostly just curious what name was given to the audio file anyway.
edited 2:47 PM EDT April 15, 2014
I have no idea how compression algorithms work, but you never know. XD
Is it possible?
edited 3:05 AM EDT September 15, 2014
Also
http://forum.xentax.com/viewtopic.php?f=17&t=3477&view=previous
Where do you see "RIFFT WAVE"? Did you manage to decompress them? If so, how?
Ah maybe they didn't compress them for SH3? Not sure which files you checked in the case above, but I don't have the SH3 section of the BD (few seeders on the torrent I found a while ago).
edited 5:00 PM EDT September 15, 2014
In SOUNDCD.IRX a table for each BGM's ID and sectors (relative to SOUND.DAT!) of both the Vers and SdDt chunks are stored. I found out that one of the values stored in the table is the ID of the BGM, because when I altered SOUND.DAT and launched the game, PCSX2's log showed an error from the game saying "Invalid Sound BGM call: 50114" which eventually lead me right to the table I needed. And since I knew an ID to look for, it was pretty easy to dissect it.
Okay, so 0x13CE0 -> 0x1431F is the region of the table. Each entry is 20 bytes in size. Struct below:
u32 bgm_id
u32 vers_sector
u32 vers_size
u32 sddt_sector
u32 sddt_size
Remember, the sector numbers are relative to SOUND.DAT, not the whole disc/image! Many of the IDs are just "1", because those are sound effects. ID 50114 is White Noiz/Forest Trail. ID 50006 is Forest (the theme playing when you meet Angela). ID 50115 is A World Of Madness/Silent Heaven. That's all I know right now.
There's some other table(s) right before it, but I don't know what the hell they contain.
Hopefully this can be used somehow to modify where each BGM plays... hopefully. Feels like I've tried everything else now.
EDIT: Damn it, guess not. But I found something else with the help of those BGM IDs.. in the main executable, there appears to be a few BGM IDs in a table (0x1D5208->0x1D5367, 8 bytes per entry), but there's only 44 there. But they're in the order that the tracks will typically play... can't figure out what the value (4 bytes) before the ID in there is though! Gah. Here's the table if anyone has any ideas..
EDIT: Okay found another little something: 0x08 in the tmid chunks is the BGM ID (this may be different for sound effects though!). Maybe I just need to edit this too to get it to properly load the BGM wanted...
EDIT: Figured out what the table between 0x137C0 and 0x13CDB in SOUNDCD.IRX is. As I thought, it's a table for the Svag streams (only dialogues pretty much) in SOUND.DAT. Each entry is 12 bytes, another struct below:
u32 svag_sector
u32 svag_size
u32 unk // terminator? (all are 0x7F 00 00 00)
Here, however, note that the sector number is relative to the end of the BGM section of SOUND.DAT (which ends at 0x19F4FFF).
Yes!!! Figured out a way to modify where BGM plays.
To change the first BGM (White Noiz/Forest Trail), change BGM ID 0xC3C2 (50114) in the table at 0x13CE0 of SOUNDCD.IRX to the BGM you want (e.g. to 0xC358 for the track "True"). Then you also patch the BGM ID for the track's SdDt chunk in SOUND.DAT (the tmid chunk specifically, as mentioned above) accordingly (i.e. once again change 0xC3C2 to 0xC358 to change White Noiz/Forest Trail to the track "True").
edited 11:15 AM EDT September 16, 2014
u32 sddt_signature
u32 sddt_size
u32 sdtr_signature
u32 sdtr_size
u32 tmid_signature
u32 tmid_size
u32 bgm_id
u32 unk1 // always 0
u32 master_volume
u32 unk2 // always 0
u32 unk3 // always 0
u32 unk4 // always 0
may be different for SFX tracks.
edited 9:29 AM EDT September 18, 2014
00 F6 90 CC 46
Not sure to what (in human-readable form), but 0x46 is the binary value in this case.
Also I think I have come to a conclusion that the raw sequences don't have any "natural" form. Some of them are simply made in such a way that only when the player progresses will they play as expected, because it seems the game really just manually adjusts the volume for each sequence pattern (or channel maybe?) as the player progresses (for example, I noticed the Silent Heaven/A World of Madness tracks can actually play simultaneously if you're right between the areas where the tracks play, which does suggest only volume is changed).
So there it is. I think if a set were ever to be created, the sequences would actually have to be changed somehow (adjust volume where needed). And even then I don't think it'd sound very well.
edited 10:07 PM EDT September 21, 2014
The first 4 bytes for each track/pattern (after the first two tracks/patterns that is, which again, specifies stuff like BPM and other track properties) specifies a channel. For example:
0x00 C6 82 C9
0x80 = channel 1, 0x81 = channel 2, 0x82 = channel 3, etc. (limited to 0x8F)
The byte right after the first 4 specifies a program to use. I'm not sure how it's changed in the middle of the track (if that's even possible), but at least at offset 0x04 of the track/pattern, it will look similar to this:
0x82 8A C0 8B DA DB 00
0x80 = program 1, 0x81 = program 2, 0x82 = program 3, etc. (probably also limited to 0x8F)
EDIT: More:
00 C6 81 C9 81 8A 91 8B E4 DB 00 FC 77 3C 6B 8A 40 CA
In this example, the value in bold specifies the start offset of the pattern. Also, it seems like 0xCA is a note-off instruction, but 0xCB seems to send a note-off instruction too, and it's typically present at the end of a pattern (before 0xFFFF), so I don't know...
edited 12:50 PM EDT September 24, 2014
So now I've taken it upon me to try and actually discover IDA and MIPS assembly in general. I have found the starting section of where it (SOUNDCD.IRX) parses the SdDt chunk I think at least, so there is yet progress... All I really want to find out though, is how I can get multi-track BGMs to play only one track (which means I'll have to know at least part of the KDT1 sequence format), and properly at that...
edited 5:03 PM EDT September 28, 2014
It reads the entire KDT1 chunk into memory before actually parsing the KDT1 chunk. So you can actually change for example "KDT1" to "XXXX" and it will actually still load it into memory, but it won't play it back (no internal error messages from the game).
Also: totally pointless, but I deciphered how it determines the end offset of the KDT1 chunk:
unsigned int get_kdt1_end_addr(unsigned int offset, unsigned int size)
{
size -= 1;
size &= 0xFFFFFFF0;
size += 0x10;
return offset + size;
}
It of course might not correspond exactly to the original source code, but it's pretty much a "literal" conversion of what IDA shows.
It uses this to check if the calculated size/offset corresponds to the size from the SdDt/sdtr header. The offset argument for the function is an address (which should be the start of the KDT1 chunk), and the size argument is the size from the KDT1 header.
As for how it reads/parses the actual sequence data, I sadly still have no clue :/ What's interesting however, is that it seems like the driver might actually support regular Midi sequences as well, because there are branching conditions for MThd, pQES, MDT1 and KDT1. So guess one could try to replace the KDT1 data with regular Midi data to see if it would play.
There are also traces of a "tvst" signature (similar to e.g. "tvag" and "tmid"), which suggests they were either trying to implement support for VSTs, or some Konami games actually do. Unless the "vst" in "tvst" stands for something else...
edited 5:57 PM EDT October 3, 2014
https://dl.dropboxusercontent.com/u/48454461/misc/sh2-wn-ds.txt
If anyone has any ideas about the other instructions/values, please do tell.
edited 6:14 AM EDT October 9, 2014
EDIT: moved it here: https://dl.dropboxusercontent.com/u/48454461/misc/sh2wnz/wnz.txt
Bonus:
I've attempted to recreate the first part of White Noiz. Here is what it might look like.
MIDI
Sample
Render
It's not perfect, but as you can see there are still a few unknown instructions/values. But we're getting there.. hopefully.
For the render I made, I had to make a custom instrument because VGMTrans refuses to convert the original instrument to SF2. :( Also, I barely tweaked anything (no reverb even).
edited 1:20 AM EDT October 10, 2014
If I had the knowledge, or some instruction on how to use it, I would be happy to help! I guess it requires you to know MIPS Assembly...?
And it really is sad that VGMtrans can't convert it to SF2, along with the SH1 instruments. :(
At first MIPS looked really cryptic to me, and I had no idea how to even begin interpreting anything. But once I learned that it pretty much just stores a bunch of values from/to registers/memory (think of registers as variables really), it quickly became much easier to understand for me, especially after I noticed IDA can show comments for each op (full names of each op basically, which is all you really need). You just gotta keep your mind into it, because you most likely won't be able to directly see changes to registers in IDA (you would need to attach its debugger to something that could run the particular program/module you're disassembling). If you're not a fan of reading 100+ pages of stuff (like myself), all I can say is you WON'T need to (not even half of that) to get a basic understanding of what's going on under the hood.
Some basic things worth noting is that in MIPS world, a "half-word" is 16 bits, a "word" is 32 bits. A "load" operation loads a value from memory into a register, and "store" stores a value from a register into memory. For example
sw r3, 0x0004(r4)
stores a word (32-bit value) from register "r3" at RAM address (4 + address in register "r4"), and
lh r3, 0x0004(r4)
loads a half-word (16-bit value) from RAM address (4 + address in register "r4") into register "r3"
And also that, when branching (conditional jump) or jumping (unconditional) to a location, the operation right after the branch/jump operation is always executed. Most other operations are really kind of self-explanatory if you just know what they stand for. addiu = add immediate (unsigned), subiu = subtract immediate (unsigned), srl = shift right logical, sra = shift right arithmetic, etc. All makes sense right? ...Well, maybe the distinction of logical and arithmetic shifts is new to you (it was for me), but there are C equivalent functions out there if you ever need to check what a certain op does to a value: http://stackoverflow.com/questions/7622/shift-operator-in-c#2463888
Here are some resources that has helped me so far:
http://www.mrc.uidaho.edu/mrc/people/jff/digital/MIPSir.html (programming-style equivalents of some ops; even if it's not necessarily R3000 assembly, it's still helpful)
https://cgi.cse.unsw.edu.au/~cs3231/doc/R3000.pdf (page 46 and 138-141)
I've started looking into SH1 instead because there are no PS2 emulators with a decent debugger (or at least I sure can't find any!) With the pSX debugger I can set breakpoints (pause the game) when it reads a certain memory address (it reads the whole KDT1 chunk into memory, and then processes it from there), so that's really helpful. Hopefully I'll find something soon...
I'm guessing you may be thinking "okay, but how do you know which address the KDT1 chunk is stored at..." For this you'll need to be a bit clever when using the pSX debugger, since it doesn't offer any way of searching for stuff. I used Memory Hacking Software to search for the physical start offset of the game memory, then I searched for "KDT1" using ASCII search within MHS. Then you take the KDT1 offset minus the start offset (since they're physical addresses and we need a dedicated RAM address). Then in the RAM window of the pSX debugger you can Goto the calculated address via Ctrl+G (remember to prepend "0x" to hexadecimal addresses, because it will parse it as a decimal address otherwise, even if it contains letters). If you've done it right, the RAM window should be at a KDT1 chunk. For example, 0x1F5600 is where the T2.KDT file (the KDT1 chunk of the first boss music) will be loaded.
I also realize now why I had to change lines per beat to 2 for the BPM for White Noiz to make sense. In at least SH1 it seems this is how it would actually determine the BPM.
unsigned char bpm(unsigned char bpm)
{
bpm &= 0x7F;
bpm <<= 1;
return bpm + 2;
}
And the MIPS code?
andi r2,r6,0x007f
sll r2,r2,0x01
addiu r8,r2,0x0002
Pretty simple, right?!
So for White Noiz that would actually mean 142.
Oh yeah, I confirmed the structure of the KDT1 header via the pSX debugger. The correct structure actually looks like this:
struct kdt1_header
{
char signature[4];
unsigned int size;
unsigned short unk1;
unsigned short rsv1;
unsigned short blocks;
unsigned short rsv2;
}
At least in SH1 the (supposedly) reserved values are never even read.
edited 12:23 AM EDT October 14, 2014
PCSX2 just got a new debugger if you didn't know yet: http://pcsx2.net/260-pcsx2-development-picks-up-speed.html
Maybe that can help the process?
Never mind, found some builds.
As I expected.. this emulator is still not quite mature. The debugger doesn't seem to allow me to break on memory reads. The breakpoint gets added but it's not in an enabled state.. But even if it was enabled, it doesn't break on memory writes either, even though they seem to be supported. :S
edited 7:43 PM EDT October 14, 2014
1: MHS and Cheat Engine only shows x86 assembly (which is harder to learn/follow)
2: there are OS/emulator calls to consider as well, so you might not necessarily follow only a particular game function's operations.
peronmls: I saw the debug version on YouTube, but I don't have it. I didn't see anything audio-related that might help. Is there anything that might help?
I wonder what this means..
edited 4:50 PM EDT October 15, 2014
Click "Скачать .torrent файл". You'll need to map two controllers, the P1 is for James and the other is for the Debug Menu.
To figure out the sequence format, I'll probably need to know more about the MIDI format or sequence formats in general. If you want to help, you can try messing around with the actual sequence data of some tracks, changing one byte at a time (make a backup of the original image!), then run it in an emulator to see what it affects (it may not be noticeable for some instructions/values though). I find it more helpful to just trim everything down to a few notes (one track only), otherwise you might not notice changes you make. The disk image offset of White Noiz/Prisonic Fairytail/Forest Trail is (((sector of SOUND.DAT) + 11999) * 2048), at least on SLUS-20228. To be more precise:
offset+0x58 = BPM and other track settings
offset+0x61 = not really sure what this is
offset+0x8B = White Noiz, part 1
offset+0x20E = White Noiz, part 2
offset+0x38F = White Noiz, part 3
offset+0x4A5 = White Noiz, part 4
offset+0x5BF = Prisonic Fairytale
offset+0x725 = Forest Trail, part 1
offset+0xA66 = Forest Trail, part 2
offset+0xDA9 = Forest Trail, part 3
offset+0xEB8 = Forest Trail, part 4
offset+0xEF9 = Forest Trail, part 5
Here are the structures known from previous research in case you need to modify them (you will need to change sizes if you add or remove any sequence data):
sddt
u32 sddt_signature
u32 sddt_size
sdtr
u32 sdtr_signature
u32 sdtr_size
tmid
u32 tmid_signature
u32 tmid_size
u32 bgm_id
u32 unk1
u32 master_vol
u32 unk2
u32 unk3
u32 unk4
kdt1
u32 kdt1_signature
u32 kdt1_size
u16 tickdiv
u16 rsv1
u16 tracks
u16 rsv2
One common convention I've noticed in sequencers is checking if certain bits are set in order to determine whether it should read another byte for a particular value. For example, (0x7F &= 0x80) would check if bit 8 is set in the value, which it is not (0x7F = [0]1111111, 0x80 = 10000000). If the bit was set, it would read another byte for the same value.
Currently I'm thinking we should first try to figure out how it determines a note value (looking at the dissected sequence data I uploaded some posts ago may help, but it's probably not accurately dissected..) If you have any ideas, post them!
I wanted to try ripping the driver for SH1 the other day, but I cannot figure out how to properly append BODYPROG.BIN (decoded) to the "main" executable in IDA so that it shows all functions properly. It detects some functions fine, but values are missing and cross-references doesn't make sense because BODYPROG.BIN needs to be loaded. If I could figure out how, IDA should also be able to detect some more "known" functions (PSYQ SDK functions that are in the "psyq" signature that comes with IDA, or even the ones Cait Sith2 made) it seems.
I may have attracted some attention to some of our threads around here. Maybe someone will come around. Maybe...
http://silenthillcommunity.com/viewtopic.php?f=20&t=436557&p=7659941#p7659941
Also, I don't believe I posted this yet? It's a list of BGM IDs and their CD sectors (relative to SOUND.DAT), paired with the track titles from the OST where applicable, or the track titles from Firebrandx's rip in other cases (...those that I could recognize anyway). The list is in the order the BGM IDs appear in SOUNDCD.IRX. It's probably incomplete since I only switched BGM IDs. The tracks would have to be manually modified (channels, etc) in order to get every part to play (and to get them to play properly) in some cases.
50000 0x00001CCD ordinary vanity
50001 0x00001D73 confronting the beast???
50002 0x00001DF5 ???
50003 0x00001E89 null moon
50004 0x00001F04 alone in the town
50005 0x00001FAB confronting the beast???
50006 0x00002015 warped heart
50007 0x000020BB confronting the beast???
50008 0x00002158 true
50009 0x0000217D ???
50010 0x000021CC ???
50011 0x00002232 pianissimo epilogue
50012 0x000022C5 promise (reprise) (unused version)
50013 0x00002355 promise (reprise)
50014 0x000023F0 daddy's revenge
50015 0x0000245A bed-head???
50016 0x000024E0 laura plays the piano
50017 0x00002512 heaven's night
50018 0x000025B5 ???
50019 0x00002639 fermata in mistic air
50020 0x0000267A magdalene
50021 0x000026B4 forest
50022 0x000026F4 ??? (unused version of magdalene/forest?)
50100 0x00002774 ???
50101 0x0000280A executioner's handywork
50102 0x0000288E ashes and ghost (unused version?)
50103 0x000028F5 ???
50104 0x0000294A mansion horrors
50105 0x000029C4 hospital predators
50106 0x00002A0F running the gantlet
50107 0x00002A69 prison of the mind (pt.1)
50108 0x00002AFB basement dwellers + another track
50109 0x00002BA2 tormented soul (???) + another track
50110 0x00002C37 ???
50111 0x00002CD2 nurse piper
50112 0x00002D55 music box melody
50127 0x00002DD7 music box melody 2 (unused?) + lakeview hotel
50113 0x00002E73 theme of laura (reprise)
50114 0x00002EDF white noiz + prisonic fairytail + forest trail
50115 0x00002F65 a world of madness
50116 0x00002FE0 the labyrinth (???)
50117 0x0000300E combing town west
50118 0x00003058 ominous hallway + butterflies & secrets
50119 0x00003099 ???
50120 0x000030D5 dueling with eddie
50121 0x0000313A ???
50122 0x00003196 ashes and tears (black fairy)
50123 0x00003203 the day of night
50124 0x000032A0 betrayal + another track
50125 0x0000333E pyramid head's lair + the stairwell
50126 0x0000334A toluca lake
50128 0x000033E9 ???
edited 3:59 PM EST November 10, 2014
https://dl.dropboxusercontent.com/u/48454461/misc/sh2bgmreplace.php
Some things to note:
- You need PHP to run this script. You can get it at windows.php.net. If you've never used it before, simply unpack it and put the script in the same folder as php.exe (usage instructions below).
- You need to configure the script (5 settings) before using it. The settings are at the top of the script. Setting it up may take a minute or two, but once it's done it should be a convenient tool to replace BGMs with (I hope).
- The script will not allow to replace multiple BGMs (i.e. running it again with different values won't work). This is because the script actually restores the original values for BGMs which aren't actually being replaced (which is everything except the two you specify of course), to avoid weird situations where the user may forget to restore BGM values when they should.
To use it, open cmd (Windows-key+R -> cmd), then:
CD /D "path to folder containing the script..."
php script-file.php
Oh yeah, not sure if anyone here is interested but I ripped the samples in an organized fashion (one folder for each track containing all its samples) the other day. https://dl.dropboxusercontent.com/u/48454461/aud/SH2Sound.7z
edited 3:30 AM EST November 18, 2014
Did you extract those VGA samples from the game? How and wherefrom?
And since you know Japanese (or are you Japanese?) Can you translate this CD's cover please? The CD purportedly contains Saigo no Uta trailer which plays a full version of Theme of Laura featuring a guitar adlib, very similar to another guitar outro adlib of one of the trailers. Here's the cover:
http://www.crimson-ceremony.net/lostreleases/item.php?id=shdemo_dvd-jpn-tgs2001autumn
And about the unused .pss video, spoilers:
Could it be the one that plays when James meets Maria at Rosewater Park? I didn't watch the file itself but that's the only bell it rings.
I extracted the VAG samples from IOP\SOUND\SOUND.DAT by using a couple of scripts I wrote, since most programs aren't capable of detecting sample rates when ripping, or won't rip to the native format...
I am not Japanese but I can try translating the text on that cover (I assume you meant the front cover).
特別制作 = Special production
電撃王 = DENGEKI Ō (a game magazine which is today called DENGEKI HIME (電撃姫) as I understand it)
2002年2月22日 = February 22, 2002
発売日は = Release date [is]
大安 = Auspicious day
士口日 = (sorry, no idea on this one, the last kanji might not even be correct, although it matches with the furigana on the black background on the cover (たいあんうちじつ) if I'm seeing it correctly)
そんなメデタイ前祝!! = Such an auspicious early celebration!!
XboxゲームタイトルコレクションDVD = Xbox Game Title Collection DVD
edited 4:15 PM EST November 20, 2014
Why would they keep the VAG samples in the game?
Aren't the sounds already saved in ADX or does the game produce some sounds using MIDI & VAG?
Did you find out how to prevent voices from playing? Most of the music FirebandX ripped had voices so he had to work around those by editing, muting voices altogether would produce a purer soundtrack.
Thanks for the translation mate!
Only the PC and Xbox versions have ADX files as far as I know. The dialogues and all music played during the pre-rendered movies in the PS2 original is streamed (SVAG/SS2 respectively).
On at least SLUS-20228, everything in SOUND.DAT from 0x19F5000 onward are dialogues. So if you want to mute voices, editing the dialogue tracks is as good as it gets. Keep in mind that some of the dialogue tracks does contain music though (an example of this is the guitar track in the "I got a letter" cutscene), so it isn't really possible to mute only the voices for all tracks.
edited 6:04 PM EST November 20, 2014
Unfortunate that dialogue contain music, FirebandX said that I got a Letter took a good share of the soundtrack's production time.
Also, what's the situation on the MIDI files? Are you able to make them? I ask this because I've really into composition recently and I love looking at how composers, especially from video games, "do" theirs. I already love that I have Kingdom Heart's SEQs/MIDIs cause Yoko is really one of the best, if not the best, VGM composers out there. Too bad not all the tracks are included in the psf2 sets (especially the Final Mix tracks)...
http://www.mediafire.com/download/goa2q0yx8257vvd/KH2FM+midi%28final%29.rar
Sorry to get so off topic. Back to SH2 research.
@mrjaredbeta: sorry, no, haven't been able to convert the sequence data to MIDI
edited 4:49 PM EST December 5, 2014
is (BGM 50009)right?
and holy shit! (BGM 50018) is the e3 pyramid head music.
edited 5:26 PM EST December 5, 2014
edited 4:00 AM EST December 6, 2014
https://dl.dropboxusercontent.com/u/48454461/aud/sh2ubgm.zip (FLAC)
Note that the last track has a few pops/clicks, which is a fault in the composition and not the recording itself afaik (I only fixed them for the video).
What's your reference as to which track was unused by the way? Did you finally play the game?
I used my BGM replacement script that I posted on one of the previous pages to change the results screen theme (which is the only reason the tracks has a small fade-in by the way). When replacing a BGM, I have to close the game, re-run the script and reload the game, since the image file cannot be changed if it's opened. Nor can I use save states since it loads SOUNDCD.IRX pretty much directly on bootup (so I used a MC save from GameFAQs). They were recorded via Stereo Mix from PCSX2 (using Audacity). If I knew of a decent alternative to burning discs (like streaming the images or something) for each BGM, I would've recorded them from my PS2 instead (which I did with the unused version of Promise (Reprise) by the way), no doubt. But I don't even have enough empty DVD discs to burn for each of these, so that's crazy talk either way.
"and did you find the original tracks too?"
By "original tracks" do you mean the tracks that were actually used? If so, yes; have a look at the sound archive I uploaded on page 5. The folders are organized by BGM IDs and track titles (from the OST and/or Firebrandx's digital-in rip).
"What's your reference as to which track was unused by the way?"
Luckily all the BGM files on the PC release are named according to the original BGM IDs, except that they are not in the 50xxx range, and the first ID is 1 instead of 0. But for the 501xx tracks, they pretty much followed it exactly, so 50100 is actually bgm_100 and not bgm_101. So it was actually pretty easy to figure out which each track was. Of course I also had the sounds used to compose each track ripped, which I could use to confirm that the PC counterparts indeed had the "same" IDs. But I DID also manually check each BGM by replacing/reloading until I had iterated each existing ID and listened to each of them (dynamic tracks will play a bit oddly if they are not intended for the current area, and this is why I decided to look up the PC release in the end to see if it could help, since it was hard to identify some of the dynamic tracks playing in the wrong places). Essentially all the tracks that were unused were just not transferred to the PC release, so there will be a missing number between some tracks (22 to 100 is a special case though - that gap is there in the PS2 release as well). Although, apparently they did put one or two unused tracks on the PC release (or so I'm told). Here's an old-ish list (the folder names in the sound archive on page 5 are more up-to-date). Those that ONLY have a question mark (i.e. without "(unused) " in front of it) assigned as the title are the tracks that were apparently unused, but still transferred to the PC release.
"Did you finally play the game?"
I have played it, but I haven't beat it. :P
edited 3:53 AM EST December 7, 2014
The list at page 5 has 50019 as '???' whereas 50018 is the one reserved for 'ominous hallway + butterflies & secrets', maybe you inadvertently misnamed 50018 & 50019 in the recording or in the list? Also, what about 50100 and 50103?
I think I heard 50018 somewhere in the trial version when I was fiddling with James' coordinates but I'm not sure. I'm trying to change his position but my Joystick is broken and P2's controls are confusing on keyboard, especially with two joysticks.
According to peronmls, 50018 is Pyramid Head's music from one of the E3 trailers (I haven't looked it up). But let me know if you find out more about it!
edited 12:25 PM EST December 7, 2014
I tried navigating it but some places prompt a crash. Fortunately, coordinate adjustment is by fixed steps so there's a possibility of mapping the Hospital level at least; however, there's doesn't seem to be any bound to the coordinates, for instance I jumped from -80400,-17200 to -700400, -17200 by coincidence with no intermediary stages.
Streaming images from a network.
I took a look at the documents you provided at the SH1 rip thread(PS2 IOP Reference & SPU2 Overview) and found the first replete with technical terms so I only read the first two chapters as an introduction, the second had a clearer view and much less abstruse. The first document states that SPU2's "work area" is 168 byte and coincidentally the BGM sectors in sound.dat never exceeded 168 bytes, is there any correlation?
Is there a way to read SPU2's Waveform Data Area from PCSX2's memory? I don't know much about how the SPU2 is emulated but extracting the raw PCM data on the run could be harder than from sound.dat. I enabled SPU2-X' Dump Memory Contents but nothing other than emuLog.txt was dumped, even though the dump files' names & location were in SPU2-X.ini.
edited 4:25 PM EST February 9, 2015
That's really awesome! Are you gonna give SH3 some love ever?
I can't find relevant info on the .iecs format inside the uploaded archive.
edited 4:29 PM EDT June 3, 2015
edited 6:00 PM EDT June 3, 2015
Can you extract the movie files from SH1 or SH2?
In SH1, I assume the video data is in the HILL file, along with audio. It turns out the TOC table in the main executable is basically for all sectors from the start of SILENT to the end of HILL. The /XA/ folder IS essentially the HILL file. However, the file sizes for the XA files aren't treated quite right by horror_x's extractor, so the extracted files are smaller than they should be (or the sizes for the /XA/ files only represent the size of the video portion, not the audio..?). I haven't quite figured it out myself yet. Anyway, you can extract the data by hand if you just go by the sector numbers that the extractor spits out (in list.txt).
Actually playing the SH1 videos is another problem though.. You can get the audio out with VGMToolbox (with its CD-XA extractor), but yeah, I don't know how to demultiplex or play the video stuff. And again, I'm only assuming it does contain video data, based on the large remaining portion if you count out the size of VGMToolbox's audio extractions, but the remainder may as well just be silent blocks or something, so I could be wrong.
EDIT: Yeah, looks like the video data is indeed in HILL. jPSXdec can play/convert them. I don't see any way to extract it as-is though.
edited 7:24 PM EDT October 1, 2015
The audio files I get for HILL via VGMToolbox are mostly small (8kbs) and cause Winamp to crash, the larger ones (~200kbs) are silence.
FirebrandX could produce a clean version of the music that plays when Maria throws away the gun, not sure how.
I asked FirebrandX how he extracted Terror in the Depths of the Fog a while ago - he doesn't quite remember it himself, but it sounded like he took it from the OST CD release.
There's an additional section at the start of FirebrandX' version of Terror in the Depths of the Fog, which has sounds of footsteps over it in the PC version. I forgot about the other percussion-centered section altogether so there was confusion on my part.
https://dl.dropboxusercontent.com/u/48454461/aud/sh2/BGM%2050003.midi
https://dl.dropboxusercontent.com/u/48454461/aud/sh2/BGM%2050003.sf2
edited 2:26 PM EST January 14, 2016
Did you happen to rip "The Day of Night" by any chance? ...either way, most definitely looking forward to more!