Silent Hill 1 (KDT) Research by peronmls at 6:55 PM EST on December 17, 2013
Well I'm slowing figuring out Silent Hills KDT track sequence format. I have found to much but I think I got as far as I could get and maybe some more advanced can figure out more. Stuff like tempo and sequence track was found but its not enough.
Here this to help:
uint Signature 0x3154444B ("KDT1") uint Size of this file in bytes uint ??? uint # of tracks for each track: ushort Size of track in bytes for each track: byte*trackSize Track event data
Sequence data is "4B 43 45 54 (KCET)" to "FF FF 00 00" The next 4 bytes of 43 45 54 4B is the size information.
It is the flag of multi song data 4B 43 45 54 00 00 00 00 54 53 44 42 00 00 00 00 "01" 00 00 00 17 byte is a multi-song "02" or more. There is a need to change the size information by cutting out the file in this case.
Must be added if there is no header above.
This is a driver for use in Suikoden 2 Edit:remove the link
Please merged the seq + vab
The File -> open in the suikoden2.psf PSFLab File -> Import Binary Data in the The merged file of Destination address 0x80010004 File -> Save As is Save psf.
I have changed the way because there is a case to merge does not work. (Position of the VAB as they refer to the size information)
Fix driver http://www.mediafire.com/download/ranulvy4yc6e7l2/suikoden2b.zip I was made to fix the position of the VAB
File -> Import Binary Data in the 0x80010028 the data from Seq(4B 44 54 31(KDT1) to FF FF) File -> Import Binary Data in the 0x8001D7C0 the data from VAB File -> Save As is Save psf.
Edit: how to merge the command-line copy command Example: copy a.seq + b.vab c.bin
This is a sample of the header http://www.mediafire.com/download/hunmwwvyd3gjbuy/KCET.zip 4B 43 45 54 "xx xx xx xx" 54 53 44 42 "xx xx xx xx" 01 00 00 00 4B 44 54 32 "xx xx xx xx" 01 00 00 00 00 00 00 00 you need part of "xx xx xx xx" rewrites the size information in the hex editor. Specified position of size 4B 43 45 54 "xx xx xx xx" Size of 0x08 after bytes 54 53 44 42 "xx xx xx xx" Size of 0x14 after bytes 4B 44 54 32 "xx xx xx xx" Size of 0x24 after bytes is until FF FF
Because PSFlab truncates 3 bytes or less, please specify every 4 bytes. size divisible by 4, Please add 00 when you do not have enough.
If you rewrite the KCET.header File -> Import Binary Data in the 0x80010004 the data from KCET.header
Import position of VAB if you specify the size of the above. the 0x8001000C + "specified size" = "Import position"
File -> Import Binary Data in the 0x80120028 the data from Seq(4B 44 54 31(KDT1) to FF FF) File -> Import Binary Data in the 0x80148000 the data from VAB File -> Save As is Save psf.
mogikihei, the problem is not the reservation for the seq block. I have already tried manipulating the headers and the locations of the KDT/VAB inside the resulting PS-X EXE and making sure the KDT/VAB data doesn't overwrite any following bytes. I tried your uploaded revision anyway, and as I thought, it still doesn't work. I really think it must be a driver issue.
foobar2000+foo_input_vgmstream returns: Decoding failure at 0:00.000 (Unsupported format or corrupted file):
It appears to be using features that are not supported in suikoden2 driver. There is a need to create a driver from Silent Hill. It is difficult for me ...
But today I got UFO.PSF to play! It turns out the track's MIDI channels are actually muted. This might be because it's pretty clearly dynamic. The first part only plays in two of the channels of the sequence, then the second part plays in another channel separately.
So it requires some (minimal) editing of the sequence data. Here is all I did: at the start of every sequence, change the succeeding byte after the first byte with value 0x87 to something greater than 0x80 (so if it is 0x8780, change it to something like 0x87C0). Yes, this will cause everything to play simultaneously, but I really don't think there's much else we can do without hacking out everything into separate sequences. :/ At least you can get both parts to play like intended if you keep the rest muted (you would have to make separate PSFs of course)...
Yes. I'm quite confident I can get everything to play, but dynamic tracks can not be played back in a traditional way (even if I did rip the original driver most likely), so I will have to make separate PSFs for each part of them. We will have to play around with the final results however we want I guess, since there isn't exactly any "true way" that some of them are meant to be played.
Actually creating these PSFs shouldn't be too hard to automate, but admittedly the delay here is actually due to me having gotten sidetracked trying to recreate one of the tracks in the game, so now I'm back at trying to reverse engineer the actual format again in hopes of converting to MIDI (or at least manually inputting stuff in my tracker-based DAW, Renoise).
I have found all functions for the driver, so basically all I have left to do is remove CD functions and any other non-audio related functions that aren't supported by PSF players. So basically I'm really close to being done. But there will be no way to play its dynamic tracks back "normally", so I will have to modify (unmute) the sequences and create individual PSFs for each of them (see the "What's going on in other xsf formats" thread).
I have also made some significant progress on a KDT1 to MIDI converter, but that's been on hold for the last couple of weeks since I've been working on ripping the driver non-stop basically.
AWESOME MAN! Yeah I wouldn't worry about muting the tracks. Unless there is 2 different tracks that use the same KDT and VAG then yeah. lol. I'm really excited for this!!
Well, it's been quite a while longer than I hoped obviously. I guess I got ahead of myself as usual.
The ETA took a turn as I realized that ripping the dynamic BGM parts were going to be more time consuming than I had hoped, because I needed to write a KDT1 interpreter to automate demuting their tracks (there's several hundreds of tracks in total...)
In addition to that, I realized after demuting all the non-standard tracks, that there are multiple songs with their own separate tracks within the same KDT (it could look like this for example: first song = tracks 2-6, second song = tracks 7-9, etc.), so I also needed to go through everything manually to determine which exact tracks I needed to rip to their own KDT/PSF file.
Also, I initially ripped the driver for SLUS-00707 (US) because SLPM-86192 (JP) won't run on No$psx due to a BIOS/copy protection check, which meant debugging SLPM-86192 would be much harder for me (since I don't really know anything with a decent debugger that runs that release anyway). But I always intended to rip the SLPM-86192 driver, because apart from it being "straight from the source", SLPM-86192 is also a more recent release and may have had some driver changes/fixes - who knows. So in order to rip that version, I had to find the same routines I had narrowed down in SLUS-00707 by comparing my IDA databases and looking for their equivalent locations in SLPM-86192. This took MUCH less time than doing the initial SLUS-00707 rip of course, but still also ended up taking a bit more time than I had hoped.
But, AT LAST I finished writing a bunch of scripts, converting to PSFs, organizing, tagging, all that good (tedious) stuff. However, I didn't time them for several reasons, but I hope I'm forgiven.
I used FirebrandX's digital-in gamerip as a reference for track transitions and whatnot, and have tried to follow the rip as closely as possible, but obviously I have been limited to what I can do with the PSFs, as everything is actually played in-place and simply gradually demuted by the game engine from what I understand. So I didn't end up combining many of the tracks after all. And I suppose there's many ways the tracks could be mixed anyway. I tried my best at least. If anything in particular needs fixing though, let me know, and I'll look into it ASAP. Also, please take the time to read notes.txt :)
Now, if I could just somehow convert SCEIVers data to a VABp container... I might actually be able to make a half-assed rip of SH2 and SH3 using this driver as well... assuming this driver supports their KDT1 data. Although, I wonder, has anyone ever used regular PSFs in a "PSF2" set (I assume it would still end up on psf2.joshw.info, since despite the subdomain's name, it seems to be for anything PS2 really)?
Finally, in case there's anyone who wants to experiment with doing their own mixing of the dynamic tracks or whatever, here is the aforementioed (prototype-ish) library/tool/whatever that I wrote to deal with KDT sequences, mainly to demute a lot of this stuff. But it can be used in lots of ways, so please don't run it unless you know what you're doing. Also, beware potentially cringe-worthy code. Among other things, there is a very basic function to print note/velocity/command events, but I could use some help with figuring out commands and improving parsing of parameters (please get in touch if you'd like to help - I can share my IDA database with loads of named functions, structs, comments, etc. for the executable). I am hoping we can combine efforts and eventually write a full-fledged KDT to MIDI converter. Currently I am trying to understand how to interpret all the time values, but I can't really figure out how to do all the conversions, so if anyone can help out with that I'd really appreciate it! The script (main function) is currently set up to demute specified tracks (command-line arguments) and write the result to a new KDT file, so you will have to do some minor modifications to get it to do whatever it is you want it to do.
Example usage: kdt-tool.py "C:\UFO.KDT" 2 3 This would demute tracks 2 and 3 of UFO.KDT (tracks 0 and 1 are always demuted if needed, as they're special tracks) and write the updated KDT data to "UFO (tracks 02, 03).KDT". You can also specify ranges of tracks (e.g. "6-9"). You might also want this to easily create PSFs compatible with this driver (just drop a VAB and/or KDT, or even a folder onto it and it'll convert to .psflib/.minipsf respectively).
If you read the full text of the last couple of paragraphs that I posted on the previous page, you'll see that I am still hoping to make a converter, but I could use some help from more "MIDI-experienced" people who has dealt with converting custom sequence data before. It seems a lot of these MIDI commands are quite similar to that of MusyX and Zelda/Mario64, so maybe if we're lucky, the people who has worked with either of those will reach out.
For now, all I have is a very basic note/velocity/command parser (kdt-tool.py, find the link on the previous page). Maybe it could help other people understand this format as well?
KDT2? Was that ever used? I know there are "KDT2" signatures in the SH2 driver code, but I never spotted any actual files with a KDT2 header signature.. ?
As for ripping SH2 and SH3 in general... I'd love to have rips of them myself, but... the documentation for ripping PSF2s are scarce. I'm particularly puzzled as to how I go about writing the PSF2.IRX module, and I have ZERO knowledge about RPC binding and all that other shenanigans that appears to go with creating such a module. I don't even know how to compile to an ELF executable, and my knowledge in C programming is quite poor. So basically the only other way I see to rip them would be to use the SH1 driver and convert the SCEIVers data from SH2/3 to pBAV somehow. Well, that, or (one day in the future perhaps) converting the sequences to MIDI and the banks to SF2.
So I think I managed to do a quick hand-made conversion (just the first couple of seconds of the Results theme) to MIDI, but it doesn't come out right when I open it in Renoise, whereas it sounds fine in Reason. Could someone who is more experienced have a look at the file? Am I missing something or is this an issue on Renoise's end?
Also, could anyone who is using any sequencer/DAW other than Renoise/Reason check it out? Does the timing match up with the original?
C.MID C.SF2 (you'll want to use the second preset/patch/instrument)
You know who really needs to know about this? Fungo over at Silent Hill Media. He's also released complete gamerips of many of the Silent Hill titles, and I think he might be interested in all your work. :)
Well, I figured out how to make playable psfs with Nisto's scripts (hint: driver.psflib needs to be in the same folder), and now I'm doing a custom mix that matches more closely how you'd hear the music in-game in certain areas.
By the way if this is something that interests enough people, I could share it publicly when it's done. It's not much, but I imagine there are some people out there who'd appreciate it.
I have a question about the Silent Hill rip, is there a way I can combine the dynamic tracks together? Can I make my own minipsf's with the tracks I want unmuted and combined?
@Moose: Yeah, ALL the .psflib files residing among other PSF files needs to remain in the same folder.
@RukarioGyiyg996: Yes, as mentioned in my original PSF post, you can use kdt-tool to assist you in the process. Use the function demute_and_isolate_all_tracks_to_separate_files to figure out exactly what each individual track is if you don't already know, then use demute_and_isolate_specified_tracks_to_single_file when you know which tracks you want to combine. As stated in the usage notes on Github, you need to make these function calls manually from the main() function currently, but it should be easy, just comment/uncomment what's already there. But please be careful, read the usage notes on Github, and know what you're doing - there's minimal error handling in this script.
To convert KDT and/or VAB to minipsf/psflib respectively: psf-make.py (as once again linked in the original post)
Yep, it should be platform-independent, considering it's Python and I'm only using its standard libraries. Either way, I'd appreciate your feedback if you try it out. :)
Assuming that Nisto means the extractor linked in this post, you don't need a XeNTaX account to get it, just follow the mirror links. Here's an IPFS mirror just in case: SHExtract.7z
Yep - support for SH2 and SH3 isn't exactly coincidental. I did look at the SH2 driver and added what few additional KDT1 commands were added since SH1. I am aware there are still lots of commands I need to support, but I believe the rest is actually only a subset of the SEQp format's commands, because the driver has a separate branch for commands not in the range 0xC6 - 0xCE. So I might simply need to have a look at VGMTrans' sources for SEQp to figure out how to convert the rest.
I am not too familiar with C/++, nor with VGMTrans' hierarchy and development structure (and GUI development would probably factor into adding that as well, which is completely alien to me), so probably not. You can submit an issue at Github and ask loveemu or whoever to implement support (they can use kdt-tool as a reference). He hasn't been active with the project for months though, so keep your fingers crossed if you do.
I would've liked to fix its SF2 converter myself, since it's broken, but C++ is such a messy language for me, and I don't even understand half of how the converter comes together. Everything is just too scattered for me to work with.