Metal Gear Solid HD Collection - .sdt files(PS3, XBOX360, PSVITA) by AnonRunzes at 10:50 PM EDT on September 18, 2016
So, has nobody cracked this thing yet?
MGS_HDC_SDT

I only have three files; one for each platform the game was released on. I'll probably bring up some more later on.

edited 11:23 AM EDT September 19, 2016
by AnonRunzes at 11:25 PM EDT on September 22, 2016
Bump.
by AnonRunzes at 12:30 AM EDT on October 3, 2016
Bump.
by vajuvaju at 11:46 AM EDT on October 8, 2016
I want to help. I own the game on Xbox 360. Can you give me some directions?
by AnonRunzes at 12:37 PM EDT on October 8, 2016
Directions, directions, directions...

0. To make sure you "own" the game, you'll find yourself with three situations:
0.1. If it's a physical copy, find a way to dump that copy through your computer. After you've done that, go to the next step.
0.2. If it's a dumped copy(.iso), use exiso. You can find it here.
0.2.1. From there on, you'll have to extract it to some folder. It must not have special characters, only letters.
0.2.2. Start the program by using command prompt. If you're confused, just Shift+Right Click on the folder you just extracted and click on something that says "command line" or "command prompt".
0.2.3. The parameter for extracting the XBOX 360 .iso file with exiso is this:
"exiso.exe -x [directory of XBOX 360 .iso file]"
It will automatically create a folder.
0.3. If the copy is already extracted, with directories and all, then you can proceed safely to the next step.
1. Use quickBMS. You can find a download link for the latest version(0.7.6a). Repeat step 0.2.1.
2. Save this script on the extracted folder.
3. Repeat step 0.2.2.
4. Type this while on command line/command prompt:
"quickbms.exe [directory of metal_gear_solid_xbarc.bms]* [directory of .xbarc file]* [output folder]**"
or
"quickbms_4gb_files.exe [directory of metal_gear_solid_xbarc.bms]* [directory of .xbarc file]* [output folder]**"
* It is possible to drag-and-drop the file into command line/command prompt.
** In case you defined a non-existent directory folder, you will be asked by quickBMS if you want to create a folder or not. Just type "y", press enter, and you're done. Otherwise, if you defined an existent directory folder, then quickBMS will proceed extracting into that folder.
5. After extracting, there will be mostly .cl5 files on the folder you specified. Repeat step 0.2.2. then type "ren *.cl5 *.sdt". You'll see that the file extension of these files had changed.

Now if you really want to help, then prove it.

edited 4:22 PM EDT October 8, 2016
by vajuvaju at 8:23 AM EDT on October 9, 2016
Done!
1) Extracted the NTSC ISO of MGSHD.
2) I want to try MGS2, so I extracted all xbarc files using the QuickBMS script.
3) mgs2_1.xbarc is the only one that had cl5 files.
4) I renamed all cl5 files to sdt.
by AnonRunzes at 11:53 AM EDT on October 9, 2016
@vajuvaju - OK, I think you're doing... well, I guess.
by AnonRunzes at 2:32 PM EDT on October 19, 2016
Bump.
by Nisto at 6:29 PM EDT on November 2, 2016
Funnily enough, here's another case (working on ZOE right now) of MPEG partitioned data with Sony ADPCM audio. Even the custom audio headers are the same as in ZOE.

However, I'm not sure how one would go about demuxing this particular MPEG structure, because it seems to start with non-audio/video headers (0x01B3 and 0x01B5 for the most part). The video stream is playable in MPC-HC at least.

EDIT: Oh, and I only looked at the PS3 file, so I can only speak for that version.

edited 6:42 PM EDT November 2, 2016
by Nisto at 8:54 PM EST on November 11, 2016
Give this a try (for the PS3 .sdt):
https://dl.dropboxusercontent.com/u/48454461/misc/sdt_demux.py

If it gives you errors, please provide some more .sdt files so I can get a better understanding.
by AnonRunzes at 9:12 PM EST on November 11, 2016
@Nisto - Just in time, eh? I'm uploading these two files that gave the tool these errors:
E:\MGS_HDC>sdt_demux.py E:\MGS_HDC\PS3\BLUS30847\PS3_GAME\USRDIR\MGS2\mgs2\us\movievr\_bp\_ps3\ending.sdt
0x0000000C: unrecognized stream ID: 30001

E:\MGS_HDC>sdt_demux.py E:\MGS_HDC\PS3\BLUS30847\PS3_GAME\USRDIR\MGS2\mgs2\us\movievr\_bp\_ps3\p080_06_m02.sdt
0x0000000C: unrecognized stream ID: 04

The former is still uploading(along with a few files to go with it), but you can find the latter on the same MEGA link.

edited 9:24 PM EST November 11, 2016
by Nisto at 12:34 AM EST on November 12, 2016
Well, looking at the new files, I'm satisfied to see that I seem to have gotten things right from the first file; it's just that there are more IDs which needs to be mapped to filename extensions. I can probably just extract anything currently unknown to *.bin, but if you require proper extensions, I'll probably need all .sdt files that you can find.

So I've updated the script, and at least for now it extracts further unknown stuff to .bin. The last part before the extension is the ID in hex. If you can put up with a bit of "hexploring", you might be able to determine appropriate extensions yourself, by looking at the first few bytes of the files. All you need to do is add new mappings near the top of the .py file (can't miss it). If you do, maybe share an update for other people that may be interested?
by AnonRunzes at 12:52 AM EST on November 12, 2016
@Nisto - I'll do my best!

EDIT: It seems that some of the files such as p004_01_p01.sdt and p070_01_p01.sdt handles a different audio codec. The MEGA folder will be updated soon with a new file(p004_p070_ps3.rar and t01-02a1-2d_ps3.rar).

EDIT2: Here is an "updated" version of sdt_demux.py. Not only does it contain new formats, it also supports formats from the PS2/XBOX versions of MGS2, and from there on, ZOE2 The 2nd Runner and MGS3. Not all of it has been covered though, but this should be enough by now.

edited 2:11 AM EST November 12, 2016
by Koto at 4:45 AM EST on November 12, 2016
So, we can expect MGS and ZOE packs in a near Future?

See ya
by Nisto at 7:30 AM EST on November 12, 2016
Question, do .pac files all have "PACB" as the magic (first 4 bytes)? In that case, I guess 0x04 should map to .pac as well. Anyway, thanks for sharing the update.

I had a look at the files with the different coding and I'm stumped myself. I tested wrapping it in an MTAF file, and all possible codings available through GENH, but I can't get it to play properly. Sorry. At least it seems like it should be possible to determine if the coding is PSX ADPCM, so that's good. I believe the byte at offset 0x0A in Konami's custom 16-byte header indicates that (0=PSX ADPCM).
by AnonRunzes at 8:42 AM EST on November 12, 2016
@Koto - Well, it might be possible but considering that most of the in-game music for some games is in .sdx format... only time will tell when the .sdx files will finally be reverse-engineered.

@Nisto - "Question, do .pac files all have "PACB" as the magic (first 4 bytes)? In that case, I guess 0x04 should map to .pac as well."
No. The .pac files are entirely different formats compared to .pacb. Just to give you a hint, these .pac files are one of those MPEG2 video formats, and it doesn't have "PACB" as the magic.

"I believe the byte at offset 0x0A in Konami's custom 16-byte header indicates that (0=PSX ADPCM)."
Well, regarding that, I think it's best to just leave it to the vgmstream staff at this point.

edited 10:48 AM EST November 12, 2016
by AnonRunzes at 12:05 PM EST on November 12, 2016
Just thought I'd leave this here:
MGS2_ZOE2_MGS3_SDT

These folders contain the .sdt files I extracted using my quickBMS script called mgs2_dat.bms which can be found here. There are two audio formats used in 0110.sdt's case, but only one of them can be converted into GENH... apparently.

EDIT: Added one file in the folder that triggered this error:
E:\MGS_HDC>sdt_demux.py E:\MGS3\SUBSISTENCE\DISC1_SUBSISTENCE\MGS\VOX\0001.sdt
0x00000030: unregistered stream / unknown header ID: 0000000F

Oh, I forgot to ask something: is there any way to do the entire batch based on a .py file, in which it processes all files at once?

EDIT2: Here's an update od sdt_demux.py. So far the only changes for this is the addition of a new container, albeit with two new types.

edited 8:53 PM EST November 12, 2016
by Nisto at 10:14 PM EST on November 12, 2016
"I believe the byte at offset 0x0A in Konami's custom 16-byte header indicates that (0=PSX ADPCM)."
---
"Well, regarding that, I think it's best to just leave it to the vgmstream staff at this point."
---
Unfortunately, I don't think there's much they can do in this case, primarily because of the fact that the developers would most likely need more to go on, in order to properly detect this format. There is no magic number in the header, and not even any known extension - it's just a 16-byte header with the bare minimum metadata.



Also, I'm feeling slightly stupid having written this script, because hcs already wrote something quite similar to my script for MGS3/4 which appears to work for the HD files you've uploaded so far as well. https://www.hcs64.com/files/demux_dat_03.zip

edited 10:17 PM EST November 12, 2016
by AnonRunzes at 11:23 PM EST on November 12, 2016
Well, I think that tool works too.
by AnonRunzes at 10:22 PM EST on December 12, 2016
Turns out demux_dat only extracts the audio files, but not the rest of the files stored within these .sdt files. Which is a stark contrast to sdt_demux.py in which it extracts everything based on the ID of each .sdt file as indicated by the header.

Oh, and may I ask you(Nisto) to have your script extract the audio files as-is(with the .sdx extension, of course) instead of being converted to GENH? Or at least implement something(like sh3-sdex in which it has two choices: to extract or build the sd.bin file) in which the user [that executes the .py file] decides if the 0x00000001 ID file should be extracted as .genh or .sdx.

------------------------------------------------

Anyway, I just wrote a quickbBMS script for handling .msf(PS3) and .9tav(PSVITA) files. I don't know how to handle the .xwma(XBOX360) files though since the dpds chunk size depends on the duration of the file it was encoded on. Here they are:
mgs_hdc_9tav.bms - it requires the func_header_AT9.bms script to work
mgs_hdc_ps3_msf.bms

There might be a few issues with these scripts though, so I might need some feedback if I ever get to work with these issues.

edited 9:31 PM EST December 13, 2016
by AnonRunzes at 9:59 PM EST on December 13, 2016
Okay, so here's a rundown of audio codecs(based on the 0x00000001 format) used in MGS2(referenced in offset 0xA):
0x00 - PlayStation ADPCM
0x01 - XOR(0x7)'d IMA ADPCM
0x02 - Unknown ADPCM type
0x11 - IMA ADPCM

There is a sample at that MEGA folder I posted that says "MGS2_MOVIE_0110.rar". The .sdt file has the 0x00010001 and 0x00020001 IDs extracted as .bin files. So I had to modify the sdt_demux.py once again to assign an extention to these ID formats(.sdx_1 and .sdx_2 respectively, to avoid confusion). Here.

edited 10:02 PM EST December 13, 2016

edited 10:03 PM EST December 13, 2016
by AnonRunzes at 3:19 PM EST on December 17, 2016
Here's a new "modified" version of sdt_demux.py New containers were added(MGS2).
by user999 at 7:52 PM EDT on March 22, 2023
Could someone reupload the latest version of the sdt_demyx.py?

Was it ever updated to batch process multiple files?

Thanks.


Go to Page 0

Search this thread

Show all threads

Reply to this thread:

User Name Tags:

bold: [b]bold[/b]
italics: [i]italics[/i]
emphasis: [em]emphasis[/em]
underline: [u]underline[/u]
small: [small]small[/small]
Link: [url=http://www.google.com]Link[/url]

[img=https://www.hcs64.com/images/mm1.png]
Password
Subject
Message

HCS Forum Index
Halley's Comet Software
forum source