Previous Page | Next Page
- by GirianSeed at 6:53 AM EST on March 6, 2017
- I looked a little into the STAGE.DAT format and found some general info. There's two types of DAT files KCEJ games use -- DATs for streaming data (BGM, CODEC, DEMO, MOVIE and VOX) and DATs for cached/resident data (STAGE, FACE in MGS2, and SLOT in MGS3). Looking into face and slot might be helpful.
First, FACE.DAT & STAGE.DAT from the MGS2 Trial Edition seem to have a somewhat different header/format from the final retail game. STAGE.DAT's header isn't encrypted here and you can see the stage names clear as day in a hex editor.
Second, FACE.DAT in MGS2 has a non-encrypted header and Solidus can even read it by default, however it refuses to read any version of MGS3's SLOT.DAT at all. Won't even spit out a header file.
Here's some stage headers and DAT files I ripped from every KCEJ PS2 game I have on-hand at the moment. It has all the versions of FACE.DAT as well as the Trial's STAGE. I also included the Guy Savage stage (cdrom0:\ABC\STAGE.DAT) and some of the smaller SLOT and STAGE files.
http://www.mediafire.com/file/4rgp36byoc2ogjc/STAGE.DAT.7z
I also made a comparison of the contents of FACE and SLOT from the versions I could actually explore and included a few notes about the HD Editions' file sorting system and stuff.
http://www.mediafire.com/file/ledx14455x56n1v/STAGE_COMPARE.7z
Also, I dunno where "MT3" came from. Might just be something SecaProject made up. Either way, I'm pretty sure MDX is the official extension. A couple .src files left in stages s19a and s19b in the PC data (both timestamped 11/18/1998 -- way before the PC port started development) use that name.
wvx1 wv000000
wvx1 wv000020
mdx1 sg000010
efx1 se000000
One last thing. Neither the QuickBMS script nor your dat-extract tool for ZOE1 works with the zoe.dat file from the TGS 2000 demo build. Looks like pretty much the same format as the final game just with a few slight differences.
http://www.mediafire.com/file/d8rr59srga0rdvu/ZOE_TGS2000.7z
Edit: I forgot to write in my notes on the HD file system that bp_streams.txt lists the files in the original order that they were stored inside of the DAT file. So for demo.dat in the original PS2 Subsistence the first five files would be these:
0000000000 us/demo/_bp/_ps3/v020_030_p020.sdt
0011601920 us/demo/_bp/_ps3/v020_040_p030.sdt
0016273408 us/demo/_bp/_ps3/v020_040_p040.sdt
0021749760 us/demo/_bp/_ps3/v030_020_p010.sdt
0024801280 us/demo/_bp/_ps3/v040_020_p010.sdt
edited 7:50 AM EST March 6, 2017
- by Missingno_force at 2:48 PM EST on March 6, 2017
- Regarding the TGS2000 version, they shifted the offsets of the filelists and filecontents by 0x18 before multiplying.
I added a dat-extract_tgs version to both the source and precompiled binary archives and modified the sdx-extract code to use the mdx extension instead of mt3, get the links from my post on page 2.
Ill have a closer look at the STAGE.DAT files later.
- by GirianSeed at 5:24 PM EST on March 6, 2017
- Hey, thanks man. Appreciate it!
By the way I accidentally labeled the second excerpt in my HD file sorting notes as bp_assets.txt instead of manifest.txt. Whoops.
- by Missingno_force at 1:58 PM EST on March 11, 2017
- Okay, so i tried getting into STAGE.DAT from MGS2 and later, but neither could i wrap my head around the header (even the uncrypted one from the MGS2DEMO) nor could i get the decryption from secaprojects ruby code to work (sucks if a required value is missing). There dont seem to be any filelists in there at all :/
As a little heads up though, i hacked together a MDX splitter that splits the sequences(?) found inside the MDX into seperate files. Once again, source and win64 binaries at the links on page 2.
The splitter supports MGS1, ZoE1 and probably MGS2 (the differences between versions are minimal, just the size of the subsequence tables). The smallest a sequence file can get is 4 bytes (just the terminating 00 00 FE FF bytes).
I already tried looking into midi to see if there are similarities, but to no success. Maybe its got something to do with the base "message" size being 4 bytes (all sequences i checked are a multiple of 4 bytes in size).
As always, tell me if you find something :)
- by GirianSeed at 4:25 AM EDT on March 12, 2017
- In the MGS3 Trial Edition from US PSM demo disc #86 in \MGS\MGS3.ELF at offset E1890 you can find the string "inflate 1.1.3 Copyright 1995-1998 Mark Adler" which hints that some kind of zlib variant was used for...something. The string isn't present in the final versions but that might just be because of how it was built. I dunno.
I also looked into the source code for Beatmania 5thMix since it was made by another team within KCEJ and some of the code was recycled from MGS1 but I don't think there's anything useful. sound.lib is precompiled and it doesn't look like the source is present (made by a different user) and even if it was I don't think the game used any sequenced music anyway. (Might be wrong there; never played it.) On the subject of compression it used some variant of LZSS derived from a file named p_decode.c from some kind of Konami source library.
None of that's probably any use here but hey I like studying KojiPro games.
- by Missingno_force at 2:04 PM EDT on March 12, 2017
- Beatmania uses sequenced music by the nature of the game (check on youtube, youll quickly see what i mean), but i assume they did their stuff differently.
Getting into the data from the MGS3 demo would require a working decrypter for the files (which we dont have right now).
On another note, do you know, if there is an application out there, that can handle wwise sequenced music (.bnk with BKHD magic) in big endian format? If not, if there is info on the format out there? When i tried searching for info on it, i found two applications which were made for little endian files and didnt work on the files from the PS3.
If this works out, ill try getting the sequencedata from the HD collection to help me figure out the data on the PS2.
- by GirianSeed at 1:56 AM EDT on March 13, 2017
- Ah. I was just assuming since most of the code I'd read for Beatmania was dealing more with XA data. For what it's worth the file \work.5th\include\SD_CLI.H was originally written by Kazuki Muraoka for MGS in 1996, though it's been modified a bit for BM. I don't think there's much to be gained from looking at it but I still find it pretty neat.
On a more interesting note though, the TGS 2000 build of ZOE1 still contains debug symbols, unlike how most PS2 games from Konami stripped them out.
And sorry, but I've really got no idea about Wwise in this case. ZOE HD's the only game I can think of that does anything that complicated with the BGM. It might be a custom setup by High Voltage Software.
- by Missingno_force at 10:46 AM EDT on March 18, 2017
- While i had almost zero progress with the mdx sequences from the original, aside from adding a switch to skip empty sequences for my splitter, i checked the wwise banks in the HD remake. Turns out that HV isnt using completely sequenced music, but uses more premade samples (like drum beats or complete melodies) with panning and reverb already applied, as opposed to the original, where they had only a few "long" samples with complete melodies, but all of them were mono with no reverb at all. I doubt ill get any useful info from the banks :/
I also checked the sequences for the titlescreen, despite only playing one sample with reverb applied, theyre pretty big IMHO (check **0000ff.sdx in root/stage/title).
I also updated the archives with the little modifications ive made since the last update, get the links from my post on page 2.
- by GirianSeed at 2:50 PM EDT on March 18, 2017
- Come to think of it, yeah, they are pretty different. I guess I just never listened all that closely because the original had a lot of phrase-samples as well. It's possible to sequence music in Wwise after all but yeah it won't be too helpful for the PS2 stuff.
By the way the *-SequenceDataBlock1.bin files look like a modification of the se0000**.efx format from MGS1. There's some FF padding but very similar otherwise.
- by Missingno_force at 5:27 PM EDT on March 18, 2017
- Thanks for pointing that out, i checked the files again and it does look like Block1 and the efx files are more or less the same.
It looks like theyre made up like this:
[for ZoE only] 0x800 bytes of unknown use (im still guessing channel management but not so sure anymore)
The next 0x800 [MGS1] or 0x1000 [ZoE] bytes are a list of some sort, each entry is 0x10 bytes long.
The first 4 bytes are a miniheader, where the second byte is the number of sequence offsets after the miniheader. Following are 0 to 3 offsets (with 0xFFFFFFFF being a blank filler) These offsets are relative to the sequenceblock of these files, which starts at 0x800 for MGS1 and 0x1800 for ZoE.
After the list come mdx-like sequences (if not exactly the same kind), however, there are no empty (as in only containing the end marker) sequences.
I updated the archives with an efx-splitter (which dumps the 2 or 3 blocks and splits the individual sequences) as well as fixing some leftovers in some of my source files. As always, get the links on page 2.
Previous Page | Next Page
Go to Page 0 1 2 3 4
Search this thread
Show all threads
Reply to this thread:
HCS Forum Index
Halley's Comet Software
forum source