Requesting a tool or process to create PSG/HES from binary by almendaz at 2:01 AM EDT on August 1, 2020
Hi, I would like to know if anyone knows some program/algorythm/process to convert these binary files to some module music format? These are sectors for the PC-FX games "Battle Heat" and "Tengai Makyou". There's no streamed sound data in both games, except their (RTF) videos, so their music must be sequenced. As of today there is not a PC-FX sequenced sound format yet, I wonder if these files could be converted to "HES" format at least. At first I thought they were MIDI, but they are not (fake/remnant header maybe?)
Well, it is MIDI (kind of, it is an archive format containing multiple MIDIs among other files) but there is some incompatible/unusual stuff going on, like an end-of-track marker right after the meta events at the start of the file, which probably cause (all?) ordinary players to simply stop there. Also, in most cases there are tons of invalid MIDI events (for example 0x3F) after the end-of-track marker. Valid events can NEVER start with anything that has bit 7 not set. Supposed key-on events make no sense because neither the key number nor the velocity (according to standard MIDI specs) can exceed the range of 0 to 127 which means those are not really note-on events but something different. As far as I can tell, the encoding is proprietary.
Thank you very much for your reply! According to what you wrote, the files must contain some "custom" MIDI - as I do not know MIDI format very much, I thought those MIDI headers were fake/misleading. The files do not seem to contain "valid" MIDI events (according to MIDI spec on the net). I found 4-bit PCM-like bytes in them, which made me think that those MIDI were some kind of tracker answer to PSX, so that's why I asked in the first place, if there is some algorythm/process/mechanism/etc. So, studying this (propietary) format seems to be the proper way to investigate sound events and their encoding mechanism (basing on your last sentence), which is way above my current capabilities of rudimentary byte-mixing. Anyways, thank you again.
The format is not entirely proprietary. The general file structure and at least all meta events at the start of the file up to the end-of-track marker follow the official specs. Each file seems to have 3 or 4 distinctive "regions" of sequence and/or other data. I have zero knowledge about the game or the soundtrack or what versions of that game exist. When I look up that game's soundtrack on Youtube, some versions sound to me like Adlib, not sample-based. It's possible that some data inside the MIDI is instrument definition/description (?) and no actual music sequence data.
So yes, if no one else already has some knowledge about the format, the only thing to do at the moment is to reverse-engineer it. I can only try to make some sense of the midi events. It could be possible they introduced custom events which for example do not need a note-off after a note-on but instead the note duration is given like in compact N64 MIDI format.
The audio files you mentioned are actually WAV files with custom format tag.
Thanks. Great insight on the files. You're right - I mistook "PCM" with "adlib-like" (opl?) instructions, too much byte-gazing on my part. I did not examine entirely the MIDI spec to make such summary as yours; there's too many events to review, more tough if they happen to be custom. The dword at 0x0 of *each* one of the 14 "tracks" (of "F*HU*401"), i.e. the first dword of the 0x44 bytes before the "MThd" magic string, seems to contain a value nearly the track bytes size (i.e. the "filesize" of the track). If OPL could be of indication, then maybe the last "region" of the tracks contains the sequence/play instructions (maybe in words/2-byte pairs ???), and just after the instrument definitions - as these seem to contain bytes that seem common/shared/same between the "tracks" (NOT MIDI-tracks as in spec), which reminded me of some "tracker" format (i.e. {header} {instrum.header} {instrum.data} {seq.bytes} ). Maybe I'm just confusing the likeness of those formats.
To me it seems the sequence data (actual note events) is the first region starting at offset 144 because it's the only region appearing to be of variable length and differs the most between files. Everything else has fixed length.
After the supposed sequence data begins a fixed-length block (can be identified by a sequence of 0x0001 0000 0000 0000) which is mostly identical between files with the exception of some bytes here and there and some kind of offset table (512 bytes long) which contains little endian DWORDs in ascending order.
I'm a huge fan of Tracker Music myself but I don't recognize the sequence format. The only formats which I instantly recognize are MOD, S3M, XM and IT. But here it does indeed look like a custom MIDI variant. At least we "know" that it uses 16 channels because there are 16 "all notes off" controller-change events in each file.
Thanks for the update. (For the same binary data) Can it be possible to convert (i.e. "force-paste into") the "sequence" data to standard MIDI (second region? i.e. data after the 0x5F bytes after "begin") just to test the sound? I cannot tell if the last "quasi-constant" region is padding/garnage data or if it's meaningful, and the differences between the 14 tracks seems offsetted by some constant for each of the tracks (maybe the offset table you've mentioned). Unless, there are not recognizable MIDI events in that region. What can you tell about the first 0x44 bytes before each "MThd" string? For reference, the binary data (container for the 14 MIDIs) is located at (LBA) sector 3760 of CD image. For now, each track structure seems to follow {PC-FX-specifig hdr (0x44)}{MThd hdr (0x90)}{seq/events (VAR)}{"last region" (0x1C5C)}
I don't know what those 68 leading "header" bytes mean. Might be unrelated to the actual file, maybe some CD sector/file system stuff like creation data, modification date, flags etc…
As for pasting bytes into the sequence: of course you can paste stuff to that location but there's nothing representing valid MIDI commands. The way those bytes are arranged is definitely not "standard" MIDI. You can tell that by certain restrictions MIDI has on how bytes have to be structured, for instance:
* a delta-value (before any event) can never have a 0x00 between itself and the event.
* because bit 7 of ANY event has to be set for it to be MIDI conform, any byte starting with 0x0 to 0x7 is invalid. A valid event will always start with 0x8 to 0xF.
* there can never be more than one 0x00 after or before an event.
* if a certain offset is supposed to be a delta value, there can NEVER be more than 4 consecutive bytes having bit 7 set. Deltas consist of 1 to 5 bytes, never more because internally the MIDI "ticks" are represented as 32-bit DWORDs. If there's indeed 4 bytes with bit 7 set and the next one has bit 7 not set, the value of that bytes must never exceed 0x0F because then it would overflow the DWORD.
* an event can never be directly come after another one without having at least one delta-byte between them.
* a parameter of an event can never exceed the range of 0..127, so anything above 0x7F is invalid.
* there are way to many 0xFF bytes in the sequence. It's of course not entirely unlikely that they appear that often but that command (meta event) is only used for initialization stuff and the "set tempo" command and stuff like that.
Thank you for the clarification and detailed exposition. I'm no MIDI expert to be able to figure this. If they went to make a custom non-sdtandard MIDI, then I bet those two "early" games would be the only ones to use that, and the other games might very well use an original, console-specific format for non-streamed sound. The information you posted is pretty useful. Please leave your exposition here as a future reference.