Procyon Studio's Digital Studio Elements' SMD format by Neubre at 8:56 AM EST on January 28, 2011
I am absolutely sick to death of this. I have exhausted Google trying to search for information on this format. A Japanese guy has apparently written a program called smd2mid, but I can't find it. Everyone seems to be complaining that more and more NDS games are using Procyon Studio's driver, but no one is doing anything about it.
So I have. I have begun work on the SMD format, specifically the version that was used in the second generation of the Pokemon Mystery Dungeon games. I've worked out all of the headers, but none of the actual music data yet. Sadly, that's the part that actually counts. D:
I think I'm going to need some help on this one. If anyone's interested, I'll post my findings.
Progress for the Procyon Studio SMD/SWD format into sequence has improved thanks to psy_commando.
I tried to use ppmd_audioutil for both 999 (which didn't work out well) and then PMD2-Sky, and after a couple tries, it led me the midi sequence, and a somewhat working SF2 (or at least for PMD2 atm).
"(!)- Exception - DSESequenceToMidi::operator(): Couldn't open file Instrument\0_B_ENV_BEACH_01.mid for writing !"
To get around this, try removing the line "out_bgm"... you'll get midi's + a SF2 file just named ".sf2" so you could rename it if you like.
The issue with this is getting other games such as 999 to work. The midi sequences will come out fine for 999, but SF2 conversion is not right... possibly because I only used testing midis (one .smd and one .swd of the same song when it probably needs everything).
EDIT: Got it to work for 999. Had to make a duplicate of the .swd file and rename it as bgm.swd. BGMs such as M05_BGM_E_01 won't convert due to no support for PSG instruments.
It's mentioned in the thread above that the pitch for some samples are off (example for 999: M04_BGM_D_04 has a D4 pitch on Sample #24 that should be D5)... although it's probably a pitch modulation problem. Same goes for M14_BGM_E_08 for channel 7 and 8.
M17_BGM_E_11 Channel #1 along with M18_BGM_E_12 Channel #3/#4 (volume problems occurs also in M02_BGM_D_02 M20_BGM_D_06 M21_BGM_D_07 [Channel #9] M24_BGM_D_09 [Channel #8] and maybe more) possibly has a volume modulator issue, that's also mentioned as a problem from the forum above.
So far, I'm happy to see and hear about this, and am loving everything from it. Fantastic work!
edited 12:58 AM EDT October 18, 2015
by psy_commando at 6:12 PM EST on December 7, 2015
Hey !
@AceK: I wish you'd have mentioned that in my thread on project pokemon! XD The first issue was probably just me being dumb and not creating the output directory I was writing into.. You tend to get a bit lazy when you have your little quick and easy debugging setup ^^;
Also, you probably want to add -log "path/to/log.txt" at the commandline so its going to output a ton of verbose and info into that file.
What game is 999 by the way? I might want to take a look at it. I've been looking for more games using DSE to gather more data, and run more tests.
"BGMs such as M05_BGM_E_01 won't convert due to no support for PSG instruments."
Honestly, I'm not even sure that the sample format the program mention as PSG, is actually PSG sound.. The format codes used for each samples just matched the ones used in nitro composer's formats, and I assumed since most matched, that they're probably the same format codes. I had never encountered a game that used that format before with the DSE driver.
"possibly because I only used testing midis (one .smd and one .swd of the same song when it probably needs everything)." The soundfont is built using the preset data stored the swd files that accompany each smd. So if you build a soundfont using a single swd, there's no preset data for all the preset that are used by the other smds and not by this one.
Also, for those interested, I wrote a lot of documentation on the SWDL and SMDL format, and how the sound driver works. PP wiki DSE hub page I had to do a lot of debugging and disassembling to figure most of this stuff, and I still haven't figured out what all the event codes do. So maybe some of you guys may want to take a shot at some of those events code ? I left a bunch of offsets and details on how music is processed on the wiki, and I also have a crap ton of notes around here that I didn't get to write on the wiki yet. I got a text file filled with the disassembly for every event handler functions laying around too if someone wants it.
And I guess I might as well talk about the issues with converting DSE stuff into midis+sounfont:
So far, the biggest issue has been about volume and volume envelopes. DSE uses a AHDSR envelope with 2 extra parameters, one to set the initial attack volume, and another to fade-out the note while the key is still being held down. Also, the volume values for each phases of the envelopes are applied using a linear function. Both the SF2 format and DLS have only support for typical AHDSR envelopes, and they both use a logarithmic function to apply the volume envelope. That's been causing me a lot of headaches.. Things just don't sound correctly with logarithmic envelopes. Being stuck with using the SF2 release parameter makes a big difference on some tracks!
Then, another massive issue is that, a lot of preset uses the built-in "LFOs" to modulate things like pitch, pan, volume, and possibly cutoff. SF2 and DLS have 2 LFOs, but DSE has up to 4.. Not only that, but DSE has support for most common waveform shapes, "LFO" with rates much higher that 100Hz(thus why the term LFO is probably not all that accurate on Procyon's part), a depth parameter that's obviously not in cents, a delay and a fade-out on the effect. Those LFO effects have a pretty radical impact on how the samples sound like, but they can't be converted to either SF2 or DLS. And baking the LFO effect into the samples would most likely sound really weird unless I render a sample for each key, which would make the soundfont enormous..
Also, the DSE format uses keygroups to mitigate issues with note stealing, and giving the devs more control. But, the SF2 format has no such features. And DLS seems pretty limited in that department..
Finally, I had a couple of question for people knowing more about audio than me. I've noticed that some samples, mainly very short ones, such as the synthwave samples in PMD2, sound out of tune when played in a track, but seem to match the root key when testing the SF2 itself.. So I was wondering if simply looping audio data couple modify the pitch of a simple waveform sample? I'm keeping the sample rate the same and everything, but it still sounds off for some reasons.. And its not a pitch modulation LFO effect changing the pitch either, I verified..
I can't figure out how the volume is calculated for the whole thing.. It seems to be ok at the preset + split level, but beyond that, everything sounds too loud. The game code is no help, since it seems to do a lot of redundant bit-shifting, and uses a ton of temporary variables in memory which makes it all really hard to follow.. If I had a basic idea of how it works it might help a bit.
Also, how do you resample a sound while preserving the loop points ? I've been trying to use a resampling library, but I need to figure out how to shift the loop points.. The few things I tried resulted in audio artifacts.. I was thinking of making a DSE music player kind of thing, that would support all the DSE features properly, but without the audio know-how its proving to be a bit too much.. -_-
by psy_commando at 5:35 PM EST on December 14, 2015
So, I added support for smdl and swdl file wrapped together in a sir0 file. Aka the format used in zombie daisuki and 999. That's probably only going to work for a few chunsoft games though.
It'll be in the next release hopefully, once I fix some other issues. I'd like to know if there are other games using DSE that would need to be supported.
And I realized I didn't need to keep the loop points if I'd just resample the sample points as I'm looping the source sample.. So yeah, derp.
I talked with ipatrix from pokecommunity and apparently that the pitch issue could be tied to the samples not being looped correctly, or at least the loop points position being off and mangling the waveform. We'll see if I can come up with a way to fix this..
Sadly I think I will be very little/no help for sequence tech since I never really looked into what flags specific frequencies and LFO's of sequences :(.
Testing 999: Nine Hours, Nine Persons, Nine Doors... m01_bgm_d_01_00000000: The smd/swd conversion has something weird with compatibility for tempo. Awave can identify tempo rate fine (unless it's default to 120 bpm) while other programs will not be able to identify and will leave it at the program's default BPM rate. This problem only seems to happen with this particular smd/swd, but the other bgms have working tempo for all applications just fine.
As for other things, m05_bgm_e_01_00000000 smd/swd (the one mentioned above with PSG not supported) now crashes the cmd line... or gives a vector error. Logging doesn't seem to work out for me for some reason, went ahead and typed the process (ignore ~~ naming headers/footers):
~~START LOG~~
<*>- Found a matching SWDL file containing samples! Exporting a Soundfont along the MIDI! <*>- Loading pair.. <*>- Exporting to MIDI + sounfont.. <*>- Currently exporting main bank to C:\VGM\2SF\Procyon\ppmd_audioutil_0.2_a\NDS_UNPACK\.sf2
<!>- Exception - vector<T> too long
#Total time elapsed: Time elapsed : 141ms
~~END LOG~~
About to look into others bgms and see how it converted. Thanks for keeping us up to date, and thank you for the work and patience!
edited 10:38 PM EST December 14, 2015
by psy_commando at 1:30 AM EST on December 17, 2015
That's alright. I'll figure it out eventually. By trial and error if it comes to that.. XD
The current 0.2 release doesn't have support for 999 or zombi daisuki. Its not out yet. There are still a few issues to work out.
But, with that said, I'm really not sure what you're doing to get that output ? In particular that line : "Exception - vector<T> too long"..
That sounds like a mangled header.. Since the files in that game aren't stored as separate smd and swd, I assume you've tempered with those to get them to work ? How did you proceed to do it ?
Also, can you post the command line you've used to run the program ?
And that's not the log output. Logging is disabled by default. You need to append "-log log.txt" to get the actual log output. Though, I doubt it will be necessary for now.
Yeah, but those are only voice clips, jingles and the sound from the movies, you can't find the music that plays normally in-game(at least I can't), sorry for bothering you and thanks anyways.
by psy_commando at 1:32 AM EST on December 22, 2015
Alright, I took another look and apparently that a couple of SMDL and SWDL are stored in "sound.pb". I know absolutely nothing about this game though, so I don't know if that's all the music.
I worked on a feature to load swdl and smdl from a "data blob" like file. And it turns out the devs might have been a bit sloppy with this one.. (definitely worse than prof layton's few missing sample references)
It also uses a lot event 0xC3 and event 0xA8, both of which are unsupported right now, because I'm not sure what they do. So the music sounds pretty bad. And the pitch bug is especially noticeable in this game. They seem to have a ton of really short samples..
So I'm not sure if I'm going to go out of my way for supporting that game right now. Because even if I did, it wouldn't sound decently anyways. Well, the soundfont export wouldn't at least.
Actually, the soundfont export will never possibly sound 100% like the original. Soundfont and DLS are both too limited to do the same stuff that DSE can do with its own "soundfont". The best would be writing a custom player. And I'm looking into that actually, but I don't know much at all about audio programming. DSE is really simple overall though.
Oh, it can't be helped then, thanks for trying tho!
by psy_commando at 3:49 PM EST on December 24, 2015
Oh I didn't say I won't support it ever, or that its impossible to do. Its just that, for the near future, I got a lot of stuff to fix/figure out first ^^;
The entire codebase for my utilities is a big messy workbench I use for research and making tools, so that doesn't help XD
But you're welcome.
by psy_commando at 2:00 PM EST on February 3, 2016
So, I just wanted to share something I just found out about. The game Luminous Arc uses DSE, but an older version of it. The version number in the smd and swd files is 0x402, while in PMD2 and the other DSE using games I found it was 0x415.
The format of the headers changes, the format of the sound samples entry in the wavi chunk, and the format of the program entries in the prgi chunk also changes significantly. I'm still investigating.
I've found another game that uses the SMD/SWD format - Ni no Kuni Shikkoku no Madoushi. The files are here if anyone wants so investigate this thing: NiNoKuniDS_DSE.rar
Speaking of the format, is it possible to develop a format dedicated to playing the Procyon Studios Digital Sound Elements format?
Btw, I figured out part of the problem with the samples being at the wrong pitch. It seems its only pcm16 samples that have that issue. I just need to figure out why.. I know that pcm16 samples have a few differences in their entries.
"I've found another game that uses the SMD/SWD format - Ni no Kuni Shikkoku no Madoushi. The files are here if anyone wants so investigate this thing: NiNoKuniDS_DSE.rar" Thanks, I'll try taking a look.
"Speaking of the format, is it possible to develop a format dedicated to playing the Procyon Studios Digital Sound Elements format?"
I'm not sure what you're talking about? Why develop a format for playing those? The format is already meant to be played?
So you guys are suggesting some kind of aggregate format? For easier distribution I'm assuming?
It should be doable then. Actually, I'm aggregating every SWD/SMD loaded + the main sample bank if it exists in my tool when I export a soundfont. So, it shouldn't be too hard to dump that.
If you mean a single DSESF file per track though, that might be a bit more involved. Because you'd need to inject the individual samples into each of those if its a game that uses a single sample bank for all its tracks. I'm still experimenting with rebuilding SWD and SMD files right now.
I'm not too familiar with the NCSF format though. From quickly looking at it, it seems to just pack SDAT files together, and apparently follows the PSF format more or less? Would it need to have the audio engine code ripped too? I think I read in another thread here that ripping the part of the binaries handling the playback and the samples and etc from memory didn't work 100% too well for some reasons. I'll have to find that thread again..
But regardless, DSE is pretty straightforward internally from having dissected the game at runtime. Everything seems to be calculated linearly. The LFO system looks a bit more complex though. So, I've been looking into writing an audio engine for playing those backs from scratch. But I'm not too knowledgeable about audio programming :/
Long story short regarding NCSF, the .ncsflib file stores the SDAT, the .minincsf files just store what track from the SDAT to play, but to actually play them back requires a player and that is what the NCSF Winamp and Foobar plugins do, they contain playback code based primarily off of FEOS Sound System by fincs.
So basically, to make something like a DSESF (or whatever it would be called), you'd need to have some sort of player code. Such a player would depend on how similar/different playback of SWD/SMD is compared to playback of Nitro Composer's SSEQs.
@CyberBotX: Thanks for the summary! That clears things up!
I'd say its pretty different from looking at the documentation for nitro composer stuff. I tried looking for similarities between the two at first, but there were only a few of them. I guess that's probably why nobody bothered reversing the format until recently.. :/ But to be honest, it was a lot simpler than I expected to reverse this, thanks to how many similarities it has to sf2/dls and midi. So its still a bit surprising nobody really tackled it before. I'd assume the nitro composer format must have been a lot more annoying to figure out.
But yeah, it really looks like DSE does more extra processing stuff that nitro composer. The volume envelopes for samples for one are AHDSR, with some extra parameters. And it has up to 4 "LFO"s per preset, each LFO supports all common basic waveforms and random noises, and can affect pitch, volume, pan, and probably cutoff(hard to tell if its resonance/Q or something like that, because the DS DSP tend to sound really "ringy" IMO). And each LFO can have a delay, fadeout, rate, and depth, and possibly something else I didn't figure out yet. Also, preset splits seem to have finer control over how samples are associated to notes and velocity ranges. And I'm guessing there's probably more differences too that I'm forgetting about.. ^^;
I'm not sure what you mean exactly about the player code? Wouldn't it need to be written almost from scratch for DSE anyways? What is the player code right now actually? Is it just something someone wrote from scratch, or is it emulated/reinterpreted ARM assembly being fed data? Sorry if that sounds dumb XD Its just because, my only prior experience looking into ripped game audio was with SNES SPCs, and apparently that those contains part of the game's audio engine, and its just emulated.
Info dump:
Here's their page with the specs: http://www.procyon-studio.co.jp/dse/spec.html (Its in japanese, but with the screenshots and google translate you can sort-of figure out what's going on)
Here are a few of the UI screenshots translated (Those really helped figuring out the missing bits) : LFO
Also the SMDL format is very very similar to a MIDI file. And the SWDL format, is very similar in concept and content to Soundfont or DLS. (The SWDL format is actually more powerful that Soundfont AND DLS.. There's so much control over a lot of little things like note stealing, and etc.. And you don't seem to have a minimum sample length for looping them.. )
Here are the wiki pages I put up on ProjectPokemon about the format. It seems to be mostly universal with other games: DSE Overview SWDL SMDL
And here are my (messy) personal notes I store on my dropbox and update as I go. Its a bit more up to date than the wiki right now, because I just began dealing with DSE files from version 0x402. (The more common version 0x415 is what's on the wiki right now) Notes
I'd also share my code, but its an horrible confused mess patched and hacked over and over again, and probably will be more confusing than helpful ^^; Its up on my github though.
Oh, and I really recommend, if anyone else ever wants to poke at the format too, to get the US version of "Pokemon Mystery Dungeon : Explorers of Sky" to experiment with. Mainly because the memory locations where most of the processing data is stored is conveniently labeled with "mseq" and "mdev", so its really easy to locate using an emulator and something like cheat engine. All other games using DSE that I tried didn't have those location labeled at all, including Explorers of Time/Darkness. Plus, all the offsets in the game code in my notes are based on that one game. AND best of all, if you get a complete savegame off gamefaqs, you get this thing called "Sky Jukebox" (Main menu->Other->Sky Jukebox), which allows you to play any tracks from the game at will, and its so very helpful!
Well, I did have help in the form of an incomplete specification for SDAT which helped me figure out most of what I needed to create NCSF. There were a few things here and there, such as the specification of the PLAYER blocks and the random/variable actions of SSEQ, that I had to do more digging on.
It does sound like you'd probably have to write some player code from scratch, or possible try to reverse engineer it from an existing game's binary. I believe that fincs said that was how he created the sequence player in FEOS Sound System, by reverse engineering Nintendo's player from an existing game. But basically, it isn't emulation. fincs' code would parse the sequence and get the sound data it needed. His version would then pass the sound data to the DS's sound system to have it play, whereas my version had to fake some of what would've gone to the DS and instead pass the sound data to the player (Winamp or Foobar).
Alright, thanks. That makes sense. Its pretty much what I was heading for with the audio engine I was working on I think.
Speaking of which, how are you "synchronizing" notes/events in real time? I haven't found a smart solution to that problem yet.. Besides making a separate thread that keep track of the time using a high resolution system clock.. But that didn't work too well..
I'm not exactly sure what you mean by "synchronizing" here. But if it does help any, here is the source code of the SSEQ Player portion of my NCSF plugin for Winamp:
kode54's Foobar plugin uses mostly the same code as this. The player part is composed of the Channel, Player and Timer sets of files. Everything else there is for loading up the SDAT.
Ah, I think I see what you mean now. I am not sure how to explain it in a simple manner, but the player fakes the DS's ARM7 clock rate and uses that as an indicator of when to process the next part of the sequence. If you look at the Player::Run() function, you can see that it uses a tempo count to handle this, which is information that comes from here:
So, I found out why most other games came out wrong using my tool.. Its just because I did a very stupid assumption, and forgot that games where each SMD/SWD pair have their own samples, don't usually share any of them..
I got confused, because all chunsoft games were doing a pretty good job at having all the same size of sample table..
So, like 4 months past the time when I wanted to release this, here's the new version of my tool.
I also fixed the issue with that 999 game. It turns out that there's a single sample whose format is 0x300, which I assumed was PSG, because the format #3 in nitro composer is PSG. But looking at its entry it seems to be actually 3 bits ADPCM samples.
Anyways, to top it off, that one sample entry in 999 is length 0.. So its probably a mistake on the dev's part. But it threw an exception. I just don't process that sample type now, and just log it. I haven't found a single example of sample data for it, so I can't do anything about it, if its even used in any games at all.
I added support for ripping matched smd/swd pairs from random formats that literally are just swd and smd continers one after the other. I call those "blobs" for a lack of better terms. It can rip things from the blobs containing music in luminous arc(snd.iear), inazuma eleven(sound.pb). It probably won't work if the game uses a main bank instead of several swd each containing their own samples. And it definitely won't work on anything compressed or encrypted. And I don't recommend using it on roms directly, because that will probably give some odd results..
I also added support for batch handling "bgm containers" which are basically just a SIR0 wrapper wrapping a smd+swd pair. That's used in 999 and zombie daisuki.
So now I can safely say that I'm rewriting this thing from scratch, because I patched over so much, that now it takes more time to find issues than it should, and its a mess. XD
I still can't figure out the issue with PCM16 samples though, I just skip baking their envelopes, and it sounds ok-ish for more tracks. But a lot of others still sound off-pitch.
And I think in general its because I have to loop a bit some samples so that they're long enough to be loop legal in a soundfont. But, that messes with their pitch a bit, especially on very short samples.
If/when I write that foobar plugin, that probably won't be an issue since I'll be able to loop the samples however I want.
What is a "loop legal" soundfont? 16 samples before and after loop point? You could just ignore that requirement. None of the software synthesizers I use care about that requirement.
@nobody1089: Its 16 samples between the loop bounds, 8 before and after loop bounds. Its in the official specs.
And I tried ignoring them, but all synths I used it on wouldn't loop. Including the bassmidi build in the foobar2000 midi plugin, coolsoft virtual midi, and the BASSMIDI software synth driver for windows. And well, viena and polyphone too. Soo, idk what you're using for playback, but, that doesn't work on anything else I've tried it on.
EDIT:Welp,actually, I think you might be right.. Sorry.. Idk what I was doing, but this time after bypassing the check for sample lengths, it seems things still loop properly.. I have no ideas why it never worked before though.. Some PCM16 samples are still off pitch though, but there's a bit less clicking now.. :/ I'll upload a patched version soon.
EDIT2:I also realized that the export without baking the volume envelope into the samples is broken. I forgot to uncomment a line after testing... And, well, I think what got me thinking that looping didn't work might have been that the volume envelope value when converted and used as-is in the soundfont sounds as if most samples were cut abruptly. I'm not sure why exactly.. When I bake the envelopes into the samples directly, I get 0 issues with the same values. But when I put them in the soundfont's generators it just doesn't work.. I assumed this far that it was because its logarithmic and not linear, like DSE's envelopes are.
And yeah, its outdated, but there's nothing else for dealing with midi playback that's nearly as widely supported. And since this is mostly something I did for fun and for ripping midis and samples. And for eventually injecting tracks and samples back into the games. It does the job.
I also knew for a while that soundfont or DLS wouldn't cut it if I wanted perfect playback, so its mostly a proof of concept, so I didn't need to find an API just to listen to the result so I could figure out the format better.
@AnonRunzes : You're welcome! Sorry about the delay. Stuff happened ^^; And thanks. Though, I probably will keeps some stuff. Just everything tied to dealing with the samples and loading stuff is really in serious needs of a cleanup ^^;
Alright, so I did the quick fix for the bugs I missed: here
Fixed the "-nobake" export as much as possible. The issue with envelope parameter durations is still there though, but at least, now samples have their rootkey properly assigned..
Also I disabled the check for loop legality. And samples are injected and looped as-is in the soundfont.
Hey! So it turns out, there's something I'm not processing correctly about the pitch correction values used in DSE.
Samples have their pitch correction data, and so do preset splits.
They both have the same 4 bytes, which represent fine tuning(probably cents), coarse tuning(semitones probably), root key, and probably what's labelled "k.tps" on the DSE UI screenshots. (Possibly "key transpose" ? it appears to indicate the difference between the root key field, and middleC/60)
But the thing is, those are fairly redundant.
Let me lay down how it seem to work:
The samples seem to have mostly default values in those 4 fields. Its usually: 0x2A 0xF9 0x3C 0x00 Which would translate to 42 cents, -7 semitones, note 60 as root key(middle C I guess), and transposed of 0 key from middle C.
But then it gets complicated.. Each splits can refer to samples by id. And they each have their own values for those 4 fields. Some may have exactly the same value as the sample. But in most cases they're not the same.
Here's an example from a swd file: Sample: 0x2A 0xF9 0x3C 0x00 Split: 0x2A 0xF9 0x18 0x24
Notice here how 0x24 is 0x3C - 0x18? The difference between the root key of the sample, and the root key of the split basically. But, I'm not sure why its even needed?
Moving on, here's another example: Sample: 0x3F 0xF9 0x3C 0x00 Split: 0x3F 0xF6 0x30 0x0C
Notice how the fine tuning value wasn't 0x2A for the sample? And notice how the coarse tuning changed from -7 to -10?
I figured out part of it. Apparently, the pitch fine tuning parameter is an unsigned byte that goes from 0 to 255, and 255 == 1 semitone or 100 cents. Or at least, from ear, it sounds about right.
After adding that little change in the program a lot of things began sounding like they do in the game, just slightly detuned. Except, it doesn't work out on its own for every samples.
There's still the "coarse tuning" parameter left to figure out. And I'm begining to wonder if its not something to pitch shift the note not in semitones, but in relation to a scale/key? Mainly because in PMD2, its usually set to -7, and its common to see scales/keys with 8 notes. And well, in the DSE UI screenshots, there's a parameter labelled k.tps/possibly key transpose. Its just a wild guess though.
From changing the pitch values live in an emulator while a track was playing, I noticed that, if I set all pitch parameter values to 0,0,60,0 the track sounds higher pitched than if I'd do the same and export the file with my program..
So I wonder if there isn't something else affecting the note pitch..
"Loop legal" SoundFonts should not mess up the pitch of samples at all. You don't need to change the loop offsets at all. Well, not much.
If the rules call for padding the start so that the loop start offset is at 16 or higher:
Pad the start of the sample data for (16 - original loop start) samples. Pad with silence. Add this count to the loop start, loop end, and total sample count.
I know the rules do call for having at least 8-16 samples after the loop end point, so:
Pad the end of the sample data with up to 16 samples. You'll want to truncate the entire sample to the loop end, then fill the end with as many repetitions of the loop data, starting at the loop start, as it takes to produce 16 samples.
Neither of these should result in audible differences to the loop. Of course, the padding should fix it on hardware and certain software synthesizers.
"The eight data points (four on each side) surrounding the two equivalent loop points should also be forced to be identical. By forcing the data to be identical, all interpolation algorithms are guaranteed to properly reproduce an artifact-free loop."
(sf2 spec v2.4)
So can't you just copy 8 samples from loop begin to the sample end, then delay loopStart and loopEnd by 4 samples?
@AnonRunzes: I made a little progress. But not much at all. I've been experimenting with pitch values a bit, and I burned myself out. So I moved onto working on the PMD2 script engine since we were making loads of progress on that. I didn't give up on it though. I'm just waiting on having an actual idea on how to figure that out ^^; And I'm probably going to do a rewrite eventually, considering things are much different than what I thought they were when I started on that utility. And with all the patching over, its hard to read or maintain.
@kode54: Thanks for the tip! I since then completely removed all padding besides the obligatory 48 zeros after a sample, and things have been going smoothly. It helped making looped samples sound a bit better. But yeah, it didn't fix the pitch.
I'm pretty sure its something I'm missing because of my inexperience with NDS audio and audio in general. I just can't seem to get the expected pitch for the samples using the values in the sound files, and I'm not familiar with how pitch shifting is done on the NDS.
Anyways. While I'm taking a break, if anyone wants to investigate on their own, I'm willing to help out in any ways I can! Any findings someone would make would be really useful.
@kode54: Sounds good! What would you need in particular though? I can rip the samples from the soundfont the tool makes, or I can just dump the samples on their own, with cue points to mark the loop points. Or, would you rather have the full soundfont? Also, I can get you some of the samples entries from the DSE sound files if you want to compare!
"Also, I can get you some of the samples entries from the DSE sound files if you want to compare!" Well, I'm pretty sure I posted a few samples on this thread, but I think it's your turn now. Feel free to post it.
@AnonRunzes: What I meant by "sample entries" was the data from the sample split info in the swd. That contains envelope info, pitch info, keygroup info and etc..
And I think what kode54 was asking for is the samples or maybe sf2 that came out from converting them?
@kode54: So, I just got you all 3 ways I can get you the samples, and I put a few sample midis where the problem is very obvious: [url]https://app.box.com/s/nfmhgu0chyi2jxghpntrpv7n01pwy2v1[/url] They should work with either of the soundfonts below.
Here's the baked samples as a soundfont, aka looped with the volume envelope processed into it, then placed into the soundfont: [url]https://app.box.com/s/s5s36hakv93dm7cnhhyrey0u1x4e1hzq[/url]
Here's the samples in a soundfont, but the envelope isn't faked, its using the envelope generators from the sounfont : [url]https://app.box.com/s/bkjwu0ozmah3lcyx28iw983d469vmpa3[/url]
And here are the raw samples as wav files. The loop points are stored as cue points in the files: [url]https://app.box.com/s/2tk5io6g9gfbrdbp6a6af6g5upeplzo0[/url]
by psy_commando at 6:01 PM EST on November 23, 2017
Just a little dredge to say I'm still looking for help with this if anyone is interested. Most of the format is figured out (Besides some metadata some events, pitch stuff and synth parameters), I just know nothing about signal processing, and what's left to figure out is mostly volume, pitch and dsp effects related really.
Its been a while since I touched any of this, but I can provide example source code and extensive documentation on the format if needed.
Soma Bringer by notveryhelpful at 12:28 AM EST on December 4, 2017
Have you tried anything with Soma Bringer for DS? It's been mentioned on this forum many times. The PS2 games Xenosaga Episode I and Tsugunai: Atonement might also have a similar driver from what I've read here. There's a somewhat recent thread with some Soma Bringer information here: https://www.hcs64.com/mboard/forum.php?showthread=51612
by psy_commando at 5:17 AM EST on December 4, 2017
Well, Procyon DSE was made for devs on NDS and Wii only. So its unlikely that the PS2 games shares anything with this format. And from what I saw in that link, its definitely not the same thing.
Soma Bringer etc. by notveryhelpful at 10:02 AM EST on December 4, 2017
OK. I wanted to make sure since I've read about Soma Bringer together with Luminous Arc etc. multiple times on the forum, and Xenosaga (...something?) was mentioned together with Soma Bringer in the thread. Procyon Studio has been Mitsuda's company for quite a long time and Xenosaga I+II for DS had already been ripped, so I assumed that the mention of Xenosaga meant Episode I since Mitsuda was the composer, but maybe it was about Episode II or something else. Do you mean that Soma Bringer is also completely different from the games you've worked with so far?
I'm well aware that I don't really have much knowledge (as evidenced by username), and that's why I haven't registered and tried to ask anything before. I also know that a lot of XS1 was played with a real orchestra etc. and the game may not even have any sequenced music, and that the company or record label is obviously not the same thing as one of its products. I just wanted to make sure that these things (Soma Bringer etc.) hadn't been brought up here because it would make no sense. Not to derail further, but here is what I've read about the sequenced tracks for Episode II before: https://www.hcs64.com/mboard/forum.php?showthread=24021 (just to make sure that this username is accurate).
Well, I just jumped into that page by accident. I just assumed that one of those "works" listed on that page out of Procyon Studios' official size indicated that they might have used some kind of middleware such as the SMD format you see now.
OK, I assumed that you were totally sarcastic since the page (http://www.procyon-studio.co.jp/works/index.html) has arranged albums ("Sailing to thee World" and such in it.
This list of games may or may not have some relevance: https://www.hcs64.com/mboard/forum.php?showthread=38229&showpage=1#post_40499
(a forum search for "Soma Bringer" brings up other DS games along with it, and this is one list of them).
by psy_commando at 10:07 PM EST on December 5, 2017
If its recorded by a live orchestra then its streamed music. DSE supports a single format for streamed music with files ending in "SAD", and with the 4 character code "SADL".
Most of this thread is about DSE's sequenced music formats though(SMDL, SWDL and SEDL). I think vgmstream already supports SAD, at least for prof layton.
I know that the live orchestral music is streamed. However, the tracks on the 2004 re-release of the album were reordered by Mitsuda as such: disc 1 - live instruments, disc 2 - synths/samples (http://www.vgmonline.net/xenosagareprintliners/). A few tracks on disc 2 sound like they might be sequenced in-game ("Game Over" for example). The album also has the following credit:
"Sound Programmed by Hidenori Suzuki (PROCYON STUDIO)"
(http://vgmdb.net/album/1799)
Suzuki was also mentioned in a thread for a sequenced DS game: https://www.hcs64.com/mboard/forum.php?showthread=38229&showpage=1#post_40503
About a similarity between DS and PS2 formats:
".SMD and .SWD files (first has the sequence data, second samples). Funnily enough, this format was used in PS2 games (Shadow Hearts 1 for instance). Just that .SWD files were Sony ADPCM then and are IMA ADPCM in NDS case (some minor differences in sequences too, of course - different hardware, but you can still see it's basically the same thing)."
Apparently, at least "World Destruction - Michibikareshi Ishi" aka "Sands of Destruction" from the earlier list should have sequences and .iear files like the ones you've mentioned Luminous Arc to have (however, the download link for the files has expired):
I've also read that Soma Bringer has maybe 2 or 3 streamed tracks, but the rest is sequenced, apparently.
by psy_commando at 7:25 PM EST on December 6, 2017
Alright, I took a look at soma, and it indeed uses swdl/smdl pairs for its tracks and it uses the same version of DSE as PMD2 aka 0x0415. Here's the first track ripped using my tool, audioutil (midi + sf2): https://app.box.com/s/jp7mgx3czje6khder2zoizszfrnlsr42
And I looked at Shadow Heart and it seems to use "SMDM" and "SWDM" files which have a different header from SMDL and SWDL.. It doesn't seems backward compatible with implementations of DSE, since the base header doesn't have the same format or a version number apparently. It would be easier if the files weren't all inside a container file..
I can say that the "0B_01.mid" file is track 1-17 "Confronting the Defense Master" from the soundtrack and sounds very accurate when played with foobar2000 with the soundfont renamed to match the MIDI file. What's weird is that the SF2 file crashes SynthFont when I try to assign it to the MIDI file. It opens fine in Polyphone though.
I tried Sands of Destruction and it works with -blobpath and the 19 MB file "01.iear", but the resulting soundfont is 265 MB.
I found out how to extract most of the Soma Bringer files as well. The tracks have sounded quite accurate, but the soundfonts definitely take a lot of space. They also crash SynthFont like before. D02.mus has failed to extract for some reason.
Is it possible that any of this could help with the ripping project (https://www.hcs64.com/mboard/forum.php?showthread=51612) I linked earlier?
I noticed that D05.mus was missing as well. Pointing it out in case you didn't already try all the tracks. I added the track titles to the filenames, and these don't work:
1-16 Amaty Ruins -Seal- (probably D02) 2-14 A Legend Sleeping in Ice Chains (probably D05)
They have the following errors:
<!>- Exception - vector(T) too long
and
<!>- Warning: The SWDL + SMDL pair #0, for preset 0 has 0 Sample info slots, and 0 samples .
Some of the tracks that work have some issues with relative volume of some instruments, as well as some other small things.
Have you encountered the message about 0 samples before?
by psy_commando at 5:46 AM EST on December 8, 2017
The soundfont format is a complete mess, so there might be some things synthfont doesn't like with my implementation. I mainly tested using Polyphone and Foobar in terms of compatibility. But tbh, I've had synthfont crash with random soundfonts before.. It doesn't seems to be very fault tolerant..
And the large size for the SF2 is normal. Since SF2's volume envelopes aren't as flexible as DSE's, I've had to render them into the samples themselves.. Which means a lot of long samples, and since its all raw audio in there, it gets really big really quick. There's a command line option to "make do" and not bake anything in the samples. That results in tiny soundfonts, but the envelopes sounds weird.
You'd definitely need a custom synth to play DSE audio properly. DLS and SF2 just can't do the same things. And, some of the filters DSE uses are basically impossible to emulate with those either.
And about the exception, I noticed some music files from games other than PMD2 seems to have a lot of quirks in their music files.. That might be why it crashes when allocating like that.. I'll have to take a look. If you get the chance, you should run it with full logging on, it might give a few more hints.
And yeah, those warnings are fine, they're there to better notice the quirks with the files. Prof layton had a bunch of instruments with no samples or anything associated to it for example. Probably just the devs being messy. Seems like soma is another one of those..
EDIT: Oh and about the volume being off, that's probably because there's basically no proper conversion done on the sample's volume. I just did a shitty cross multiplication to convert the volume's proportions. But the volume on the NDS hardware is stored differently than it should in the SF2 format, so it would need a proper conversion. But I basically knows nothing about signal processing, so I can't really do much about it right now.
The logging for the broken files just stops before the part that says "Writing Soundfont..." in working files. I didn't notice error messages in the logs before this happened either, so I haven't included other parts for now. It's possible to get MIDIs from both of the broken files separately to confirm which track was which. These are the points where the logging stops for me at least:
D02.mus (1-16 Amaty Ruins -Seal-), 14th preset:
====================== Handling Trk#0_prg#0xf ====================== --- Split#0 --- Remaping DSE( atkvol : 0, atk : 0, dec : 0, sus : 127, hold : 0, dec2 : 127, rel : 55 ) Skipping Attack.. Skipping Hold.. Decay was 0, falling back to decay 2.. Handling Release.. Remaped to (del, atk, hold, dec, sus, rel) ( -12000, -12000, -12000, -12000, 0, -2067 )
---------
D05.mus (2-14 A Legend Sleeping in Ice Chains), 11th preset:
====================== Handling Trk#0_prg#0xe ====================== --- Split#0 --- Remaping DSE( atkvol : 0, atk : 0, dec : 0, sus : 127, hold : 0, dec2 : 127, rel : 58 ) Skipping Attack.. Skipping Hold.. Decay was 0, falling back to decay 2.. Handling Release.. Remaped to (del, atk, hold, dec, sus, rel) ( -12000, -12000, -12000, -12000, 0, -1684 ) --- Split#0 --- Remaping DSE( atkvol : 0, atk : 0, dec : 0, sus : 127, hold : 0, dec2 : 127, rel : 127 ) Skipping Attack.. Skipping Hold.. Decay was 0, falling back to decay 2.. Handling Release.. Remaped to (del, atk, hold, dec, sus, rel) ( -12000, -12000, -12000, -12000, 0, 6041 ) --- Split#0 --- Remaping DSE( atkvol : 0, atk : 0, dec : 0, sus : 127, hold : 0, dec2 : 127, rel : 58 ) Skipping Attack.. Skipping Hold.. Decay was 0, falling back to decay 2.. Handling Release.. Remaped to (del, atk, hold, dec, sus, rel) ( -12000, -12000, -12000, -12000, 0, -1684 ) --- Split#0 --- Remaping DSE( atkvol : 0, atk : 0, dec : 0, sus : 127, hold : 0, dec2 : 127, rel : 50 ) Skipping Attack.. Skipping Hold.. Decay was 0, falling back to decay 2.. Handling Release.. Remaped to (del, atk, hold, dec, sus, rel) ( -12000, -12000, -12000, -12000, 0, -2752 ) --- Split#0 --- Remaping DSE( atkvol : 0, atk : 0, dec : 0, sus : 127, hold : 0, dec2 : 127, rel : 50 ) Skipping Attack.. Skipping Hold.. Decay was 0, falling back to decay 2.. Handling Release.. Remaped to (del, atk, hold, dec, sus, rel) ( -12000, -12000, -12000, -12000, 0, -2752 ) --- Split#0 --- Remaping DSE( atkvol : 0, atk : 0, dec : 0, sus : 127, hold : 0, dec2 : 127, rel : 50 ) Skipping Attack.. Skipping Hold.. Decay was 0, falling back to decay 2.. Handling Release.. Remaped to (del, atk, hold, dec, sus, rel) ( -12000, -12000, -12000, -12000, 0, -2752 ) --- Split#0 --- Remaping DSE( atkvol : 0, atk : 0, dec : 0, sus : 127, hold : 0, dec2 : 127, rel : 127 ) Skipping Attack.. Skipping Hold.. Decay was 0, falling back to decay 2.. Handling Release.. Remaped to (del, atk, hold, dec, sus, rel) ( -12000, -12000, -12000, -12000, 0, 6041 )
---------
I can post more of the logs later if it makes any difference for this.
by psy_commando at 6:51 PM EST on December 8, 2017
Definitely seems like some dev weirdness.. I'll take a look at the SWDLs for those tracks and try to see what's different about them..
by psy_commando at 4:15 AM EST on December 14, 2017
Thanks a lot for the quick fix. However, D05 appears to play the wrong sound at some points. For example, at 0:40, 0:44, 0:48 etc. there's a jingle bell sound (Prg10->Smpl0) that is replaced with a drum roll (Prg10->Smpl1) when the MIDI file is played. A single drum hit is played instead of the drum roll as well, from 1:01 onward.
Edit: These sounds are part of Prg10->Inst10, and Track 14 in the MIDI file has these instruments. The MIDI file has the jingle bell mapped to G5, but the soundfont has it tied to F5 (this note is not actually used). The MIDI file also has the notes G#6 and A6 in Track 14 but there's no sound associated with these in the soundfont.
Maybe the crash in earlier versions might be related to this, since these sounds are part of the soundfont preset (0xe) where the logging stopped.
edited 7:04 AM EST December 16, 2017
by psy_commando at 10:13 AM EST on December 17, 2017
Alright I'll look into that. To be honest, I wouldn't be surprised.. This happened before, and my code handling sample mapping is a complete half-assed mess.. However, I didn't notice anything wrong with other games.. So it might be a side effect of some of the samples that this preset refers to not existing. I noted a few programs referring to non-existant samples.
EDIT: Ok, I hear the drumroll too. I'm surprised its just limited to that though. If its what I think it is it would have shifted a lot of instruments.. I'll definitely need to look at this more closely.
Other sounds in that preset have been switched as well. I edited the soundfont with Polyphone to make it more accurate, but something (like the order of Smpl2 and Smpl3) may still be wrong.
There's also a rattle sound in some samples after SF2 conversion. The oboe (Prg2) in this track is a good example. I read through your notes and couldn't find a mention of this, so I'll say it if you haven't encountered it before.
Edit: The rattle could be present because the loop points are slightly off. It's possible to create a similar rattle for example by exporting D01 with -nobake, going to one of the looped samples in Polyphone and changing the loop start or loop end value just a little.
Also, D05 doesn't work with the -nobake option enabled: <!>- Exception - invalid map<K, T> key
It may be related to what I posted earlier since the logging stops at the 0xe preset again.
Thanks for all the effort.
edited 9:09 AM EST December 18, 2017
by psy_commando at 11:15 PM EST on December 23, 2017
That's probably when I changed the way samples were assigned to keep track of their original sample IDs..
I've heard the rattle before yeah. I could take a look at it.
I think I'll need to make a huge rework of this program at this point.. Because I feel if I fix this, I'll just end up with other issues because of the way things are handled right now. The biggest issue is how I have to abstract things when converting from DSE to SF2, so that programs still refer to the right programs and samples. Its extremely hacky right now.
This isn't important or anything, but I'll ask about it since I was going to post something anyway.
These are the OST track numbers for the files in case you're interested in comparing them to the tracks in the YT playlist or something like that:
0_W02 = 101 Soma Bringer 0_W03 = 102 The One Who Soars Over the World OPENING = 103 Ring 0_W01 = 104 Prologue - Land of Promise - S_TITLE = 105 Determination 0_E11 = 106 The 7th Division of the Pharzuph 0_W04 = 107 Myrtle - Knowledge of the Journey - 0_D01 = 108 Junel, the Large Forest 0_B04 = 109 Threatening Yell 0_E09 = 110 A Memory That Cannot Be Pursued 0_C01 = 111 Schildkröte 0_C02 = 112 Amaty, the Ancient City 0_C07 = 113 Country of the Main Faith - Guiding Sound - 0_F01 = 114 Green Wind 0_D02 = 115 Amaty Ruins - Seal - 0_E13 = 116 Tender Flow 0_B01 = 117 Confronting the Defense Master 0_E24 = 118 Movement of Unrest 0_C03 = 119 Ordital, the Maneuvering City 0_E23 = 120 Drifting Anxiety 0_F02 = 121 Run Through Flames - Largo Bridge 0_E21 = 122 Signal of Emergency 0_E08 = 201 Idea 0_D03 = 202 Graave Stronghold 0_E22 = 203 Footsteps of Stupor 0_E16 = 204 Sacred Relic 0_C04 = 205 Maran Asa, Country of the Main Faith 0_F03 = 206 Adel Desert 0_E14 = 207 Hidden Prediction 0_D04 = 208 Temple of the Sand Coffin 0_E17 = 209 Umbras 0_F04 = 210 Snowy Foot of the Mountain 0_D08 = 211 Bonfire in One Hand 0_C05 = 212 Village of Snow 0_D05 = 213 A Legend Sleeping in Ice Chains 0_E19 = 214 Relic of Ancient Times 0_B03 = 215 Assault 0_F05 = 216 Watching the Tower in the Sea of Flames 0_D06 = 217 Ring Tower 0_E18 = 301 Key of Light 0_C06 = 302 Kremona, the Sky City 0_E20 = 303 Seeking the Road That Sinks in Illusion 0_F06 = 304 Zain, the Floating Continent 0_D07 = 305 Soma Palace 0_E26 = 306 The Lamplight That Floats in Tears 0_B05 = 307 Advent - Time of Judgement - 0_E31 = 308 Decision 0_B02 = 309 Alertia 0_E04 = 310 Light of Truth 0_E05 = 311 Unleashed 0_E10 = 312 The Bonds That Return to the Sky 0_E06 = 313 Fragment of Thought 0_E07 = 314 Looking at Trouvere 0_D09 = 315 Exploration of the Secluded Land EXCLEAR = 316 And the Tale Is Spun
by psy_commando at 11:49 AM EST on December 26, 2017
Yeah C07 seems to have a very long release time on its samples. And release is not very well supported at all. There's a half-assed placeholder calculation for it in place right now, so it doesn't sounds too weird.
I'd need to figure an equation to convert release from DSE to SF2's timecents, and I'm not sure how to get that. I don't really know what DSE's release parameter is measured in..
I noticed that the program doesn't export all the files in the game Sands of Destruction since the names don't match fully (they may have Japanese characters since the names are all garbled):
<*>- Found container off: 0xDBF550, 24 Æ’tÆ’BÂ[ƒ‹ƒh( , of type smdl ! <*>- Found container off: 0xDC01B0, 24_Æ’tÆ’BÂ[ƒ‹ƒhÂH , of type swdl ! <*>- Found container off: 0xE6D530, 49 ‚¿‚т‚܎ë‚è , of type smdl ! <*>- Found container off: 0xE6F670, 49_‚¿‚т‚܎ë‚è , of type swdl ! <*>- Found container off: 0x10679B0, 07 Æ’â€Æ’BÆ’gƒ“ƒŠƒe , of type smdl ! <*>- Found container off: 0x1068A00, 07_‘“‚Ì“sÆ’â€Æ’BÆ’g , of type swdl ! <*>- Found container off: 0x1178210, 10 â€ÃŸâ€šÂµâ€šÂ¢â€™J.SMD , of type smdl ! <*>- Found container off: 0x1179270, 10_â€ÃŸâ€šÂµâ€šÂ¢â€™J.SWD , of type swdl !
etc.
These are in root\data\01.iear like the tracks that work.
edited 6:19 PM EST December 29, 2017
by psy_commando at 10:47 PM EST on January 1, 2018
Interesting.. Its the first time I see sets of SMDL/SWDL that don't have matching names..
I made a quick push with a commandline parameter that makes the program match smdl/swdl in a blob by order instead of names. Since the matching swdl is right after the track's smdl in this case.
Just add "-nmatchoff" to the command line and it should try to match by order instead of by name.
@psy_commando: How feature complete is your exporter? Good enough to just use for the DS games without proper 2SF packs? Or should I wait for improvements?
@Knurek: Well exporting tracks to midi won't ever be perfect, mainly because the filters and volume envelopes differ significantly from what's possible with soundfont.
I haven't yet really had time to look into making a full standalone implementation of a foobar2000 plugin or add to existing plugins.
So, if you really require accurate rendering, you better run it with the emulator really. Unless you just want to have the note data for a song as a midi, then export is pretty much very accurate.