MGS4 ReRip by Ultrafighter at 4:52 PM EST on December 12, 2018
Hello guys, I think this thread is gonna be 1 half of a ripping guide / tip collection for anyone who'd like to continue my project of reripping Metal gear solid 4 Guns of the patriots & 1 half of my own progress report in case I manage to do everything by myself. As you may remember I pointed out here (look for the last post on this page for original message) that the rip by !!!!! which I reuploaded and mirrored might be buggy / bad because at least 3 tracks ended very abruptly and with minutes of silence where it shouldn't be. Anyway I wasn't really planning to rerip this game myself at that time.
Well recently I stumbled upon this topic and decided to test author's BMS which should support MGS4. It didn't work out too well - not only the script is very slow it unpacks everything without extensions & doesn't output readily playable tracks with expected MTA2 headers. If extracted files are edited afterwards (a portion of data from file start is deleted so that there's an "MTA2" string at offset 0x0) and saved with *.MTA2 extension they inevitably crash a player with VGMStream installed.
I searched a bit and discovered Konami DAT Utility by Sonix from this tool pack but it couldn't open any of my sample *.DAT archives (screenshot).
Finally I learned there's a program by HCS and that was quite a breakthrough: as soon as I started using demux_dat_be.exe on DATs it became obvious that this utility was used a lot while making !!!!!'s rip because naming scheme was the same. I still wasn't satisfied with results as an executable stopped dumping files to HDD wherever it encountered any unusual / untypical info and that left me with very few MTA2 tracks unpacked from DAT bigfiles (like 300MB worth of files from a 2GB bigfile). Here's what it reports on such occasions: movie.dat & demo.dat errors.
1 more thing: it looks like original ripper didn't deal with a single demo.dat (that notorious 12.3 GB monstrosity which is well known on the Net), he seemingly found / downloaded / made tinier demo#.dat (not less than 17 such files) and used demux_dat_be to rip lots of audio streams from those custom archives. Could he use Simple File Splitter or 7-Zip splitting tool? Those progs can divide any given file into multiple parts with the same sizes for each part (only the very last one is normally smaller). It would supposedly trick demux_dat_be.exe into believing there's nothing wrong with input data and let it skip some troublesome parts of a huge initial archive (demo.dat but maybe movie.dat as well) but it should also lead to broken files (for example the ones with lots of silence in the end, the ones I mentioned earlier and put here for anyone to inspect) and it's highly likely. Even if he did precisely that and it sorta worked I don't want to follow his example & make another faulty rip, I gotta find another, better method!
At this point I have 2 main courses of action - a) to contact author of that BMS and suggest that he improves extraction of archive contents or devises a way to process unpacked tracks to make them ultimately playable by vgmstream OR b) to turn to the man behind demux_dat_03 tools and ask him to look into DAT samples & possibly update his executables so that they're able to dump all MTA2 streams stored in DATs (my main focus is demo.dat + movie.dat but there might be 2-3 more bigfiles I'd like to unpack correctly). That's all for now, Ultrafighter over and out!
The movie.dat error seems related to a 32-bit file offset limitation (as I'm using unsigned long), this needs to be 64-bit. I don't know what to make of the demo.dat error, though. It's possible that it would also be fixed with large file support.
I don't have a setup where I could work on this or even deal with those files for a few weeks, though. Should be a simple enough fix for someone with C experience.
by Ultrafighter at 3:52 AM EST on December 14, 2018
It would be awesome if it was indeed the case! And it looks like you don't need samples to attempt such a fix (ATM)? I thought I had to make some cuts from both DAT files and the offsets where your EXE stops should be somewhere in the middle. Although it's not anything really urgent it'd be great if you reported any progress here, I'd really appreciate it. Cheers!
by Ultrafighter at 2:15 AM EST on December 15, 2018
Hey that's a lot of options, I can probably resort to Nisto's solution whenever necessary so thanks for pointing it out! Actually there's already 1 more solution to unpacking MGS4 DATs: I asked AnonBaiter A.K.A. AnonRunzes to further improve his BMS and he wrote a new version, it works on all files but that pesky demo.dat. Of course I have yet to actually listen to everything it extracted but so far so good.
BTW I've already found a proper variant of "(cut) movie.dat_00013_0001.mta2" from this pack, it should probably indicate there're no broken tracks this time. Cheers!
by Ultrafighter at 3:34 AM EST on December 22, 2018
You see QuickBMS outputs 3 huge files ranging from 3,98 GB to 2,70 GB and they inevitably choke both demux_dat_be program & mgs2_stream script; maybe someone can divide them into tinier files and make either an EXE or a BMS work?
I've done everything here but some files from bgm.dat can't be extracted. It only gives me An error occurred while writing to file (The encoder has terminated prematurely with code 1 (0x00000001); please re-check parameters). Any help?