New music format for sequenced DS music: NCSF (potentially to replace 2SF) by CyberBotX at 2:09 AM EDT on March 27, 2013
Hello everyone. As the subject of this thread states, I have made a new PSF-style format for Nintendo DS sequenced music. I am calling it NCSF, short for Nitro Composer Sound Format, due to the fact that it uses the SDAT files made with Nitro Composer from the Nintendo DS Nitro SDK.
Full details of the format, as well as the tool(s) to make them and the Winamp plugin to play them, can be found at the NCSF Home Page.
Current listing of files will slowly grow. The process of making a good NCSF set is time consuming, although the time to create the files is much faster than making a 2SF set, and NDS to NCSF includes an automated timer, avoiding the need for sseq2mid.
I'd like to give special thanks to fincs for his work on FeOS Sound System, a library for his Nintendo DS operating system, FeOS. The library's code was adapted by me to play without the DS, and if it wasn't for this library in the first place, playing the sequences would've taken a lot more work. I also used small portions of code from DeSmuME, so thanks go out to them as well.
While stated on the home page, I'd like to point out my reasoning behind creating NCSF. While I like the idea behind 2SF, I dislike the overhead needed by the DS emulation of DeSmuME, as well as noticing that 2SFs made via VGMToolbox still write the entire SDAT to the final 2sflib (I know it "cleans" them, but the process of cleaning them is only to zero out the unused data, the file size is unchanged).
While NCSFs can have some minor playback issues (I am unsure whether this is an issue with my own code or with FeOS Sound System), I feel that it can eventually replace 2SF.
As an aside, I had to make a small change to VGMToolbox to get it to allow me to tag the minincsf files with it's xSF Tag Editor. I cannot recall where the change was, but if the demand for NCSF grows, I will work with snakemeat and any others who work on VGMToolbox into incorporating NCSF within it. I do not wish to have a 100% automated NDS to NCSF creation tool like the current NDS to 2SF tool works, but I think mostly automated with some toggles to allow for more manual operation would work better.
This is neat. While I don't see myself redoing all the sets on JoshW archive, when the time comes for pmh to do NDS sets, unless some outstanding bugs will be find with this approach, I can see this becoming the main standard for NDS rips.
I don't really mind the high level approach, we've been using this for ages, both for VGMStream and module related formats (and PSFs, I guess, what with both davironica and Mark Grass drivers).
Please give some examples about the playback issues so far, I haven't tested much, but can only say that the games that didn't work with CaitSith2's ripkit (Touch Detective, Neves) work with yours.
Some things you might consider re: NDS2NCSF:
a) I'd switch the -t behaviour, to have it enabled by default. I don't think many people will bother to RTFM, and if you output untimed sets by default, prepare for some mails/messages asking why the rips play for 2:00 for all files.
b) I'd also suggest to round them to full seconds - it currently outputs timers with 1/1000th of second resolution, which isn't really needed, I think. So, 1:37.6667 should be written as 1:38, 1:24.2222 should be written as 1:24 (saving of 4 bytes per minincsf).
c) The current trimming method could be improved - it isn't really all that great for larger SDATs. You can either go full auto, and have the tool generate thousands of SFX files in some cases, or manually toggle them by hand. I haven't really played with the exclude/include options, but would it be possible for the program to generate a (preferably SMAP) text file, listing all sequences, let the user edit that by hand (removing lines with unwanted items) and then just read that from NDS2NCSF and output only the files listed? It's what I've been using for the non-VGMToolbox 2sf sets, seems to be the fastest way to trim them.
d) I know you aren't really the person to ask this, but are there any chances for support for other drivers? Especially Procyon Studio (Layton 2-4, Soma Bringer, Luminous Arc, Shiren 4-5, list goes on) - since it's used in some of the best NDS games music-wise. There are at least few dozen games that don't use NitroComposer for music, would be nice to have a way to listen to them as well. Let me know if you need more information
e) I'm all in for adding STRM/WAVE/ADX/AHX/HCA extraction capabilities to the tool. Loads of games, even those that use mostly sequences for music use them for title themes - adding filerippers for them should be a 10-20 minutes job - just reuse VGMToolbox schemas if you need.
f) Speaking of streamed tracks - is there any way to demux the ActImagine/Mobiclip videos? Many games have their opening/ending tracks embedded in those movies - while I can just record their audio with Desmume, it'd be much better to have them in their native format.
As for tagging, I don't think there's any need for special VGMToolbox modifcation, psfpoint should work on them fine, right?
//EDIT
Ah, one more thing - you might want to look at building an option for 2SF->NCSF tag export/matching, akin to the stuff VGMToolbox does for 2SF(UF version)->2SF(CaitSith2 version). Might save a lot of time with tagging files.
a) Are you doing any SWAR optimization? Some games (Ninja Gaiden Dragon Sword for example) have only one large SWAR file and SSEQ/SBNK pairs which use it. The SWAR file is 8MB in size, only a quarter of which is used by the music SBNKs. For proper size optimization, you'd need to rebuild the SWAR file and remove the SWAVs not used by any of the selected SSEQs.
b) On that note, some games would benefit from selective sample removal from SWAR as well. For example, most music tracks from Rune Factory 3 have a water flowing sample that is always on (and I imagine, dynamically switched on/off by game logic). Would be nice to actually be able to listen to the game without that noise on top.
c) A bunch of games uses NitroComposer files for music, but don't keep them in SDAT container. Off the top of my head, Tengen Toppa Gurren Lagan required me manually building SDAT files with a hex editor. Since you do some SDAT rebuilding with your tools, how about adding a simple SDAT builder as well (merges SSEQ/SBNK/SWAR with the same filename from a dir into a single SDAT, nothing else is really needed, since you can then merge those with NDStoNCSF)
d) There are games that use invalid volume values for some of the tracks - usually setting it to zero. Rips done with VGMToolbox allowed for manually overriding this value to play those songs - does your tool do that as well? Or is that irrelevant with your method? Try Sonic Colors SEQ_glad
e) Same with tempo command override - look at Mario Kart DS 2sf rip for example - it has two version of most race tunes (normal and sped up)
e) Last on the agenda, how do you handle dynamic tracks? I recall Star Fox Command having some of them, maybe Animal Crossing: Wild World as well. Do you allow for splitting them into separate minincsf files?
And with your current code, DSI enhanced games will have wrong suffix - they use TWL instead of NTR. Not sure if you can detect that and modify the prefix automatically, if not no big deal, since it can easily be done by hand.
At the moment I don't have time to go into depth on all those, but I'll come back to these tonight and look them over more thoroughly. But thank you for looking into it.
I look forward to seeing some nice previously unripped stuff from this.
I've updated VGMToolbox in SVN to handle NCSF in the xSF tagger, XSF2EXE, the xSF Recompressor, and anywhere else xSF formats are handled generically (thanks to the nice generic nature of the PSF format and OOP interfaces). See r927. I'll update the download when time allows.
I know for a fact that Phoenix Wright: Ace Attorney: Trials and Tribulations wouldn't play with Caitsith2's 2SF driver (mostly the music exclusive to that game and not the music from the prior 2 games in it). But those songs play perfectly find through NCSF.
Re: a) I think this is a great suggestion. I might even change the -t flag so it takes an argument instead, with 0 meaning not to time and any other number being the number of loops to time for.
Re: b) This is also a good suggestion. Chances are I might go the route of always rounding up, though. I'm not sure about that yet. But I think extra time might be slightly better than less time.
Re: c) I can say that you might be right about this, since Sonic Rush Adventure was an absolute pain in the ass to work with. It has 32 separate SDATs in it, and most of them contain only 1 or 2 actual sequences with the rest being dummies that are named the same as actual ones in other SDATs. But I do like the idea of dumping to an SMAP and reading that in later for trimming purposes.
The main goal behind the trimming/merging was to make it so only a single minincsf was needed. On top of that, I didn't want to go down the route that VGMToolbox took, as I described earlier. I also didn't want to keep around sequences that were actual duplicates. Elebits: The Adventures of Kai and Zero, for example, had the same song repeated 11 times. The sequence itself was only in there as 1 file (and referenced 11 times), but it also contained 11 duplicate banks, and those were not separate files.
Re: d) I never really looked too hard into the non-SDAT sequenced formats, but I know that Luminous Arc's music is actually streams, in SAD format, but with a different header than what Professor Layton had. And PL2, from what I had seen, uses Procyon's sequenced format, which I have no idea about it's format.
Re: e) STRM is easy to add support for, since those are already in the SDATs in most cases (and hell, just searching for them instead of using the SDATs would work too). I'm sure searching for other streamed formats wouldn't be hard either, as long as they have a distinct header with size information on them.
Re: f) I've never tried doing this, and I suppose it would depend on how the audio is stored in the movie file. I can't say how easy/hard this would be without looking at one of the files, though.
Re: tagging, I didn't need VGMToolbox to tag the files, but I find that psfpoint works nicely for batch tagging multiple files with the same tags while being harder to work with for tagging individual information. I do like the suggestion of a 2SF->NCSF tag copier, I'll look into figuring this out.
Re: Knurek's second post
Re: a) I am not doing any SWAR optimization. I could look into doing so, but that would probably have to be an additional step that the tool does, as it would have to "play" the music and determine which waveforms are actually being used, and then after that, it would have to modify the SSEQs and SBNKs accordingly. I had toyed with the idea of optimizing the SBNKs when I was initially making the stripping tool, but I decided against it at the time just because of needing to edit the SSEQ.
Re: b) Hmm, perhaps for something like this, a minor edit to the specs would be in order, possibly to allow for the minincsf to edit the SDAT in the ncsflib, similar to what most of the other xSF formats do. I'll give that one some though.
Re: c) This could be done. And while I haven't tested it, I think NDS to NCSF actually will take a single SDAT as input, although there is a possibility that it'll give a strange output filename for the ncsflib if there happens to be a valid country code stored in the byte where the game code would be in a DS ROM. Although I doubt that would ever happen.
Re: d) I've not checked out the music from Sonic Colors, but I'll do so in a bit. I make no changes to any of the volumes coming in from the SSEQ itself. So if the sequence really does set the volume to 0 for that track, then it'll be 0 on playback as well.
Re: e) This might be another reason for the spec change I mentioned earlier.
Re: e2) I think it would depend on how they are laid out in the SSEQ, but currently it only outputs 1 minincsf for each SSEQ.
Re: the country code thing, I had copied those from VGMToolbox, so it is also wrong about those 2 country codes. I can fix it in NDS to NCSF, though.
Re: DSi detection, I'd probably only do this if there is an easy way to detect that the incoming file is a DSi ROM and not a DS ROM. Even the current methods don't check if the incoming file is a DS ROM, it just assumes that it is and tries to read the game code regardless.
Re: snakemeat
Thanks for doing that so quickly! I'm not sure if the change I had made to my local copy applied to all those tools (I had only been using the xSF Tag Editor with the NCSFs), but it's awesome to hear that you've made all those other tools handle them as well. :D
Re: JFD62780
I'll have to download XMPlay and see what happens. As for Sinc interpolation, I'd have to look into the algorithm for that. The current interpolation was copied from DeSmuME, which only had Linear and Cosine interpolation in it. But since I'm not limited to that, I can look into other forms of interpolation as well.
I know for a fact that PW3 wouldn't play with Caitsith2's 2SF driver (...) But those songs play perfectly find through NCSF.
Yeah, I noticed that your approach nets more playable files, but what I'm actually interested in are examples of songs that don't work with NCSF currently -> While NCSFs can have some minor playback issues.
I might even change the -t flag so it takes an argument instead
As long as it defaults to 2, no problems with me. :) I also don't see any problems with applying ceil() function to timers.
Sonic Rush Adventure was an absolute pain in the ass to work with. It has 32 separate SDATs
Have fun with Pokemon White/Black - ~3000 sequences IIRC, 90% of which are SFX. Should take a few seconds to trim with SMAP file, but manually selecting all BGM entries (without an undo function, I should add) will take... dunno, 10-20 minutes, I guess. 2sf->ncsf matching should be really easy to add, especially if you stick with CaitSith2 format only - both mini2sf and minincsf should only have a matching sequence number, so all you really need to do is unpack them and compare. :)
Elebits: The Adventures of Kai and Zero, for example, had the same song repeated 11 times. The sequence itself was only in there as 1 file (and referenced 11 times), but it also contained 11 duplicate banks, and those were not separate files.
Please take caution with that, there are games that use the same SSEQ with a bunch of different SBNK to have the same song play with different instruments. At least a binary compare should be done on all suspect SBNK (I didn't read the source, so I can't say if you're doing that already). Preferably SBNK+SWAR parsing and then binary compare on all data used by each song variation (SSEQ+SBNK+SWAVs)
but I know that Luminous Arc's music is actually streams, in SAD format
Uhh, no, there are a few SAD streams, true, but most of the game music is sequenced. Procyon Studio format, matching *.smd and *.swd files. Both are completely different than NitroComposer stuff, and as such would require loads more effort/skill to play than I have. I'll upload some ActImagine/Mobiclip examples later. Both should just use IMA ADPCM, but it's muxed with the video stream.
STRM is easy to add (..) (and hell, just searching for them instead of using the SDATs would work too).
But you loose the filename information then. I'd stick with SDAT extraction for them. :)
I am not doing any SWAR optimization. I could look into doing so, but that would probably have to be an additional step that the tool does, as it would have to "play" the music and determine which waveforms are actually being used, and then after that, it would have to modify the SSEQs and SBNKs accordingly. I had toyed with the idea of optimizing the SBNKs when I was initially making the stripping tool, but I decided against it at the time just because of needing to edit the SSEQ.
Please note that many games (especially those made later in systems lifetime) use the same approach as Ninja Gaiden Dragon Sword - single SWAR file with all instruments/SFX. That game is the extreme example though - it has only two sequenced music files, totaling at around 1MB of instruments, rest of the SWAR file is SFX. NCSF rip could be optimized by around 90% size-wise if you'd add SWAR rebuilding.
That said, I don't think you need to parse/change the SSEQ in any way to do the optimization.
For each ripped SSEQ, find the matching SBNK. Parse SBNK, note which SWAR files it uses (up to 4 can be used IIRC), then note down instruments used for each SWAR file (that data is in SBNK AFAIR). Repeat for all user-selected SSEQ files, then rebuild each SWAR used - just null the unused SWAVs, so that you don't have to touch SSEQs).
It's a lot of work, but as you said yourself - you don't want to have files larger than required. This is the biggest size optimization you're not doing already that I can think of.
I make no changes to any of the volumes coming in from the SSEQ itself. So if the sequence really does set the volume to 0 for that track, then it'll be 0 on playback as well.
The volume data I'm talking about is in SDAT header, the SSEQs themselves do not set it to 0.
INFO block of SDAT,
typedef struct tagSDATInfoSseq { u16 fileID; // for accessing this file u16 unknown; u16 bnk; // Associated BANK u8 vol; // Volume (...) } SDATINFOSSEQ;
For 2sf ripping, changing that value from zero to, uhh, default value (can't recall what it was ATM) fixes some unplayable songs. Not sure if it's needed with your approach.
DSi detection, I'd probably only do this if there is an easy way to detect that the incoming file is a DSi ROM and not a DS ROM.
According to this byte 0x1C0 in the header not being 0x000000000 should do the trick. Haven't verified versus actual NDS files though. Like I said, don't spend too much time at this, it's easily fixable with any mass-rename capable tool.
I'll have to download XMPlay and see what happens.
Easily reproducible - the whole player crashes in in_ncsf.dll as soon as you stop (not pause) the music. No problems like that with vio2sf or any other winamp plugins.
Odd thing is, when I mentioned that some NCSFs were not playing correctly, my example was track 2 of Time Hollow, Fragment of Memories. I don't know if I had fixed something since I made the NCSF rip, but there was a note that sounded like it was getting cut off early. But after I applied ReplayGain to the tracks and checked it again, it sounded identical to my current 2SF of the song. So I have no idea where it got fixed.
At the moment, I am actually doing a full binary compare of every file when checking for duplicates, so if 2 duplicate SSEQs use 2 separate SBNKs, they will not be considered duplicates.
I'll still look into SWAR optimization, but I'm not going to put it at a high point on my list of things.
Originally I was going to use the vol value from the INFO header for the SSEQs, but I found some songs just were not working properly when utilizing it, so I decided to just not use it at all. I can't recall the example, but I think it was one of the tracks from Ace Attorney Investigations: Miles Edgeworth.
I'll look into the crash, I'm sure it would probably also affect my 3 other plugins that I've made which use the same framework (I have ones for SNSF, GSF, and 2SF, all based on the vio*sf plugins [with SNSF being based on Caitsith2's plugin as well], which is why I haven't released those ones yet).
I forgot to ask, CaitSith's rip are playable on real hardware (verified that myself), yours will not work though since the playback routine is high level, is that correct?
I don't know nothing about nothing, so forgive me if this is an ignorant question: will all future revisions of the Winamp plugin, or other players for this format, require the relevant Visual C++ etc files by necessity? Or is that just part of the early development?
Knurek, you would indeed be correct about that, since NCSF only contains the SDAT and not a Nintendo DS ROM.
I should mention this, I updated the SDATStuff.zip archive earlier, timing is on by default now (2 loops by default) and the -t flag will take an argument of number of loops, with 0 meaning to not time at all. I also made the fade lengths arguments on the command line, with the defaults being 10 for looping tracks and 1 for one-shot tracks.
The next update I'll be putting out for the tools will have better handling on finding if files were in the previously made NCSFLIB, as well as making it so it only deletes the files from the destination directory that were made by NDS to NCSF in the first place. I should've done that before, but just imagine how annoying it would be to have something like a .zip of the music in the destination and then it gets deleted when you re-run the tool. I imagine no one wants that behavior. :P
I'm currently working on a tool to copy tags from 2SFs over to NCSFs. Mainly because it'll be damn useful but also because I realized I didn't want to manually identify and tag Pokemon Diamond/Pearl. But that will probably be done in the afternoon. I'm actually heading to bed after I finish this post.
Dais!, it's just the C++ Runtime that it needs. That is only because I am compiling the plugin and the tools with Visual Studio 2010. With the way the code stands now, that is the minimum version of Visual Studio that it will compile with. But the compiler isn't needed to utilize the plugin or tools, just the Runtime, which is linked in the enclosed readme files. I could make it not require the runtimes, but the side effect would be that they would have much larger file sizes than necessary, so I doubt I would do that.
I've asked a friend if he could take a look at SDAT optimizer (STRM/SEQARC removal, unused/duped/user selected SSEQ/SBNK/SWAR trimming, SWAR rebuilding), will post the tool here when/if it's finished.
Apologies if I sound a bit demanding, it's just that I have ripped some of the NDS games three times already (MID+SF2, 2SF V1, 2SF V2), so I'd need to do that 4th time, I really, really would prefer to have it as good as possible. :)
Honestly, I'm impressed at what you have already done, and I'm really glad that this approach works.
d) There are games that use invalid volume values for some of the tracks - usually setting it to zero. Rips done with VGMToolbox allowed for manually overriding this value to play those songs - does your tool do that as well? Or is that irrelevant with your method? Try Sonic Colors SEQ_glad
So that may explain why Theme of Sonic Colors is not playable at all.
Oh, something I forgot to mention. I did try loading in_ncsf inside of XMPlay, and then attached Visual Studio to the process and stopped a track. The crash is odd... it seems almost like some of the memory has been deallocated, but not all of it, since it crashes inside of the playback thread even after I've stopped the track. XMPlay also never seemed to call the stop() function in the plugin, which may be part of the problem, but I have no idea what sort of black magic XMPlay uses to handle Winamp plugins.
So although I had planned to have the 2SF Tags to NCSF tool done yesterday, it took longer than I expected, especially squashing the crashes and bugs I was running in to. But it's ready and in the ZIP file now. NDS to NCSF also has those updates I mentioned yesterday. Next thing I'll probably be working on is to have some pseudo-SMAP support as was suggested. The rest of today is a bit iffy, so I might work more on that tomorrow.
I downloaded the 2SF set of EO3 and then made an NCSF set of the game from ROM. I played both in Winamp and they start off exactly the same. I'm not sure if that means I've fixed something in in_ncsf in the last few days or if my own version of in_2sf is flawed. But whatever pop there is, it sounds like it's there in both the 2SF and the NCSF. It may just be a tad bit more pronounced in the NCSF.
I've updated my plugin, seems you sneaked in a hidden update ;), but the problem persists. Might be something to do with me using XMplay, but I couldn't get the plugin to work with Winamp, so can't compare.
I've uploaded comparison wavewrites - this is how it sounds for me, really pronounced in the NCSF version, unnoticeable in the 2SF version.
//Edit
Oh, and if you want to take a look at how the SMAP can work, take a look at Snakemeat's SDATOPT, should be up somewhere on the VGMToolbox page.
Hmm, are you putting zlib1.dll into the Winamp directory or the plugin directory? Needs to be the former, Winamp is kinda weird about where extra DLLs go. Otherwise I have no idea why it wouldn't work in Winamp, since that is what I use. What version of Winamp?
I'm still not sure what would cause the pop. It's pretty hard to debug the actual music itself, since you can't run the debugger and listen to what is playing at the same time. I also don't know if the issue would be with how I get the sound from the SWAVs or if it's a problem with the FeOS Sound System library that I adapted for this.
As for the SMAP thing, I was going to see how VGMToolbox does it and do something similar. It might be slightly different in terms of what filenames will show up, but otherwise, I'll try to make it look close to the same.
XMPlay's author responded to the plugin crash query:
When you press "stop", XMPlay will ask the plugin via its SetOutputTime function to seek back to the start, ready to play again. The plugin's Stop function unloads the file, so that isn't called until the track is unloaded, eg. when you press "stop" again. In this case, the plugin is crashing shortly after the SetOutputTime call. In fact, any seeking backwards is resulting in a crash here, and it's happening with Winamp too.
Hope this helps a bit.
The EO3 issue seems likely to be a problem with sample decoding. Since VGMStream supports SWAV, you can just compare your output to the stuff from here and see if there are any differences.
That helped a lot, actually! I wasn't properly handling the case of seeking backwards with the NCSF plugin. I've fixed it now, and neither Winamp nor XMPlay crash when I seek backwards or when I stop in XMPlay. I've put up an updated version of in_ncsf now.
As for sample decoding, I'm not sure. I did notice in my early testing that some of the ADPCMs that I decoded sounded a little different compared to how VGMStream plays them, yet in most cases, even with that, the final SSEQ output sounded correct for the most part. I've looked over the code for the IMA ADPCM decoding and I can't see anything wrong with it. The only other thing I can think of is that it might have to do with how I set the starting sample when setting up the note for a track.
So I put up a new version of in_ncsf just now, the main thing is that I added 3 more interpolation techniques to it. It now contains B-spline, Hermite, and an "optimal" algorithm. I got them from Olli Niemitalo (http://www.student.oulu.fi/~oniemita/dsp/deip.pdf). It may not be the Sinc interpolation that JFD62780 asked for, but he said it was good enough for him. :P
Oh, and I did find a song that currently doesn't play correctly with NCSF. The opening for Kirby Super Star Ultra, while it plays fine in the 2SF set, doesn't play correctly in the NCSF set. It sounds like it's missing an instrument or something. I haven't dug much further into it yet to see where the fault is.
I'm amazed that the plugin has matured as quickly as it has. Now that the plugin's matured, I'm issuing a Call To Arms, as it were; The NCSF library, as it is, is too small. Not to mention, the only game I'm personally familiar with (Chrono Trigger) don't quite have the magic of the SNES original (though the bonus tracks it does have ain't that bad).
In short, the format needs more soundtrack releases. Don't believe me? Listen to a soundtrack with Optimal mixing and tell me that don't sound Studio Quality! ;D
(tl;dr, I want the Castlevania games, Planet Puzzle League, Sonic Rush and Contra 4, plz! (I kid.))
EDIT: Just to be clear, I do NOT want all the work to be solely dumped on CBX; he had enough on his plate creating the format! Why, he created, listed, and is hosting, the tools in its creation on his site. So what are y'all waiting for? >;)
Not to be too much of a dick, but what are you waiting for then?
CBX has provided his ripping tools and the process isn't more complicated than using 2SF ripkit/VGMToolbox.
To reiterate - this is not something like custom driver PSF/GSF/USF, or older console rip formats which required technical knowledge (and a lot of work) to do. Just grab the NDS game you like to have, run it through CBX tool and you're done (baring any playback problems, like with that Kirby track).
There's really no need to involve other people here. Really :)
I'm quite happy that you made this. I like the new interpolation methods, which really help eliminate the DS crackling and nastiness. I also like the sample rate options. These are both things I'd wished was possible with the current 2SF plugin, so it's great that you've fulfilled that wish.
There is one thing I have to ask about though: the output volume is higher than with the 2SF plugin and can cause audible distortion. Reducing the volume in the tag and in the plugin configuration don't help because they reduce it after it's been mixed together and the damage has already been done (this is also a problem that occurs with 2SF amd many other plugins for xSF formats. I think PSF is the only one where turning it down in the tag does prevent clipping). Would it be possible to implement the volume change during the mixing rather than after and prevent clipping?
"CBX has provided his ripping tools and the process isn't more complicated than using 2SF ripkit/VGMToolbox."
Yup. Not that hard. Literally just drag and drop. Roms aren't that hard to get as well.
On the other hand, the new format won't change the way the tracks sound...so if there was trouble emulating a particular soundtrack as 2SF, it'll have the same issues as NCSF :)
About the volume thing. So in the INFO section of an SDAT, there is a vol variable for sequences. As it stands, I'm ignoring that value. But I get the feeling that I shouldn't be ignoring it. I know that it's 0-127, with 0 being nothing and 127 being max. But what I don't know is when it should be applied. I had tried at one point to set the player's master volume to that, based on something fincs mentioned to me. That was a bad idea because it caused tracks to play incorrectly. The only other thing I can think of is to either apply the volume to each channel individually before mixing them together or to apply that volume after the mixing is done. If anyone has some insight as to the proper handling of that variable, I'd appreciate it.
As for the comment about the way tracks sound, while most NCSFs will sound the same as their 2SF counterparts, as I mentioned, some NCSFs play incorrectly in comparison to their 2SF counterpart. I still need to dig into this further to find out where the fault is, it could be something I did wrong in handling the samples or it could be an issue with FeOS Sound System itself, which I would have to consult with fincs about. I'll check it out better over the weekend here.
Why not try mixing the channels internally at 32 bits, and scaling the results back to 16? Wouldn't that give you enough of volume range to handle any distortion.
Not sure if Winamp API allows for that though, but might be worth looking into.
Well, the mixing is being done at 32-bits, it's being clamped to 16-bits before being stored in the buffer that Winamp gets. Some volume changes could be done before the clamping, but others, like ReplayGain, are done after the clamping has taken effect.
Are we talking 32 bits fixed point or floating point? If it's floating, could it just be output like that without reducing to 16? If so, lowering the output volume level should allow us to avoid the problem and make ReplayGain's prevent clipping function more meaningful. Or whatever way Highly Experimental handles it with PSFs would be a good model.
I don't know how the inner workings of these things function, so I apologize if those are ignorant questions/suggestions.
No, I'm talking integers, not decimals. Winamp expects signed 16-bit integers for the data, so that is what it gets. The actual per-channel samples are in 16-bit PCM format, so I add the channels together into a 32-bit integer and then clamp down to 16-bit before putting it into the buffer that Winamp will eventually get.
Thanks for clearing that up, CyberBotX. And HUGE thanks for working on this format! :)
I was also a little curious - isn't vio2sf basically the same thing that's used in DesMuME? I was surprised to find that DesMuME plays the sound to certain games more accurately than vio2sf does. Any idea why that might be the case, or whether it could be improved with FeOS?
I'm sure that things can be improved. fincs has told me that he does have the disassembly for the unimplemented commands, so it's just a matter of him getting around to implementing them. As for any other issues, I'd probably have to make sure the issue is with FSS itself and not my modified version of it.
I really wouldn't know why vio2sf would play things less accurately than DeSmuME, since vio2sf is basically a stripped down version of DeSmuME. It's possible that it's because vio2sf might not be using the most up-to-date version of DeSmuME, but that is just a guess without looking at the code for it.
I just put out a minor update to in_ncsf. This fixes crashing if Winamp tries to access a file that no longer exists, Caitsith2 reported this to me. It also makes it so the SSEQ's volume from the INFO section is being utilized. I'll probably have to redo the ReplayGain on all the current NCSFs on my site as a result. I also did a little minor optimization.
I also think I realized what happened with the opening track of Kirby Super Star Ultra. I haven't checked this completely, but I believe that there are 2 versions of the opening track, one which plays correctly and one which doesn't. I can only guess that the incorrect one uses an invalid bank. I'll have that NCSF set updated when I update the others.
I'm still working on updates to the actual tools, I just haven't been feeling great lately. Other than the SMAP feature, I may also be adding in a way to handle DSiWare, courtesy of Caitsith2. I'll probably have this done in a few days.
Although the 2SF->NCSF tag changer doesn't seem to work with KSSU 2SF rip, output mentions not being able to find the SDAT in 2SF if that helps you with anything.
The correct track still sounds slightly different than the 2SF version though, nothing major but it's definitely noticeable. I'm not sure which version is correct - this may as well be the fault of Ys driver used by 2SFs...
The new volume method is a great improvement - a 6.55dB decrease on the track I compared! Obviously that gets rid of quite a bit of the clipping, but it still can occur, although it hasn't been audible to me yet in the few tracks I've tried. I think the only way to be able to stop it from happening entirely is to be able to alter the volume going into the mixing. You're now at least on par with 2SF though, yours actually ranging being just slightly quieter to much quieter on the handful of tracks I tested. So even if you did nothing at this point your plugin wins in my book for the better interpolation.
Aside from the crashing you've now fixed, the only other bug I've encountered is that sometimes optimal interpolation causes loud clicking on some tracks for no apparent reason. B-spline doesn't and sounds almost exactly the same. "At the Pub" from FF Tactics A2 is a good example of this behavior.
Knurek, what is odd is that I ran into that problem with the FFTA2 2SF rip but not the KSSU 2SF rip. One thing I've done and will put out with the next update of the tools is that the rename option of 2SF Tags to NCSF will try not to overwrite a file if it happens to find a duplicate. Instead it will tack on something like "_Duplicate" with a number.
nothingtosay, can you tell me when in that FFTA2 track it has the loud clicking? The thing about the optimal interpolation is I used the 4-point, 5th-order version of it from that PDF I linked to, but I used the one designed for 32x oversampling. Since I have no real way of knowing how much oversampling/undersampling is going on with the SWAVs being played at different sample rates, I used the highest I could find, but maybe I should use something in the middle, like 8x or 16x. Another alternative is to use one of the 6-point interpolation methods, but I am kinda against that if only because it causes more calculations to be done which can slow things down and might cause buffer underflows in Winamp. I already run into them when I use a sample rate higher than 96000 when I'm debugging, since the debug version is going to be slightly slower anyways.
marcusss, technically, the plugin appears to work fine in XMPlay as it is, since XMPlay can use Winamp plugins. As for foobar support, I may look into that eventually, I tried to design my xSF framework so I could (hopefully) migrate it to foobar2000 without too many changes.
Well, the error turned out to be less deterministic than I thought. I took the first 12 tracks from Tactics A2, to get a decent sample size since it doesn't happen on every track, and rendered them to WAV with the output plugin. I've tried writing the file several times with different settings, but the only way I've managed to get it to click again is with volume set to .5 and sample rate at 192kHz, which, maybe coincidentally maybe not, is how I had it when I first got the clicking. But it doesn't do it every time and it doesn't do it in the same places. Repeating the test the same way twice more gave me no apparent clicks (I checked the waveforms to see instead of listening; they stand out high above anything else). I couldn't even get it to do it at all on "At the Pub" as I had before, but I did on "Gathering Allies" and "Cid" once each.
I doubt it will help you, but just so you know I'm not crazy, here is a short clip demonstrating it in "Gathering Allies". Perhaps if anyone else encounters this they could post here their settings and on which track. Maybe it doesn't only happen on optimal interpolation, that's just the only time I got it.
As it is, I doubt this is even something needing much priority since it would appear to be so rare and random.
About this potential 6-point interpolation, that would give better sound, right? My understanding is the more points the better, like SPC's sinc interpolation being 8-point and clearly better than the other options which are 4-point. I think if you have the ability to do something better, you should include it if it's not too much trouble. Keep the current scheme and add the more complex, potentially better one, and maybe add a note that it is slow and could cause dropouts. Some people's computers may be able to handle it, or if somebody wants to convert it to another format you might as well do it at the best setting you can. That's the way I'd handle it if I were doing this, anyway. Just my two cents.
Yeah, definitely heard it in your sample. I haven't tried it here myself, but I'll dig into it more later on. I do like the idea of giving people the option between 4-point and 6-point interpolation. You are correct that more points could potentially lead to better sound. I'll look into adding at least one of the 6-point interpolation options.
OK, I put up a new version of in_ncsf. Mainly because I went a little overboard and also partially because I'm sure the audiophiles will like this, I added a few more interpolation options. Mainly added in Legrange interpolation and a few variations of the Hermite, B-spline, and Optimal interpolations. I also dropped the Optimal interpolation down to the 16x oversampling version. In addition, I also changed how the interpolation handles data near the start and end of the SWAV sample. Before, I was using the slope to extrapolate the non-existant points outside the range, but now I am just using the current point for any points outside the range. Hopefully this prevents some of the clipping/popping that may have been occurring. One other thing I did was make it so changes to interpolation and muted channels will apply immediately after clicking OK instead of on the next song play.
I'm still working on updates for the tools. My sleep schedule has been horrible lately so it's been affecting me a bit more than I imagined. I'll put out an update once those are done.
Thanks for the options and the updating without needing to restart playback. I think I may have discovered an issue with the new start/end method. Sorry to keep raining on the parade, haha.
Here are clips of a section of a track from Mystery Dungeon: Shiren the Wanderer with the previous version of the plugin, the latest, and the difference left after canceling out all that is the same between them. You don't even need to use the difference to hear the clicking that has been introduced in the new version. This is the only track I've found with this problem so far, but I haven't listened to a lot of stuff yet since this is rather time consuming to do. This was done with the old plugin set to optimal interpolation and the new one set to optimal, 4-point, 4-th order. It happens whatever sample rate or interpolation you choose and is definitely caused by the interpolation because when you have none on it doesn't click. This is not random like the other clicking I found either; it does happen consistently in the same place every time.
Question about the oversampling: I doubt 16x vs. 32x makes an audible difference, but why do you think the higher number may have caused the previous type of clicking?
I have no issues with problems being brought up, that's why I made it public in the first place. :P
Hmm, maybe it's my speakers, but I didn't hear clicking in either one, and heard absolutely nothing out of difference.flac. So that's a bit of an odder one...
The only reason I thought that changing which oversampling version of the optimal interpolation would cause a difference is because I don't know how much oversampling is being done. I'm not really 100% familiar with how to calculate oversampling, though, so if someone knows how to determine that, I may be able to put some conditional code in place to use the best oversampling for an SWAV depending on the difference between it's sample rate and the playback sample rate. Otherwise I'd have to just pick a version of the optimal interpolation that would try to cause the least amount of problems. If the latter has to be done, should I go back to the 32x version or stick with the 16x version or use one of the lower ones like 8x or 4x or 2x?
Sorry, should have mentioned it's quiet. Here is the difference file with the volume increased by 26 dBs, making it easy to hear and see the clicks in an editor. But I wouldn't have bothered bringing it to your attention if I couldn't hear it with the new plugin at my normal listening volume though. Maybe I just listen to music at a higher volume than you, because it's sort of faint but definitely there to me. Starting at 4.75 seconds you should be able to hear rapid clicking, especially in the left channel.
I'm not too familiar with oversampling myself, but I see no reason higher rates should cause sound issues when lower wouldn't. But as I said, I doubt 16x versus 32x is even audibly different either. If you include options for both settings with optimal interpolation, I'll do some tests to find out how different they are. But the new clicking I found was either caused by the new 16x oversampling or by the new interpolation start/end handling. My bet is on the latter.
OK, so I put together a page to show off all the interpolations in Olli Niemitalo's PDF, plus the Cosine interpolation from DeSmuME. It's located here.
Basically, from what I see, the interpolation methods that actually go through the original points are: 2-point 3rd-order Optimal 32x (except that it looks strangely like Linear at that point...), 4-point 2nd-order Watte tri-linear, all the Hermite interpolation, all the Lagrange interpolation, all the 2nd-order Osculating interpolation, Cosine, and Linear. The others do not go through the original points, which can potentially cause things to sound quieter than intended and possibly even different than intended.
So the surprises. I had copied the code exactly as it was in the PDF for every interpolation method. 2-point 3rd-order Optimal at lower oversampling rates has skipping at each data point as well as getting closer to looking like Linear interpolation by the time it got to the 32x oversampling rate. 4-point 2nd-order Optimal and 6-point 4th-order Optimal go way out of control and I'm not sure why.
Another thing to note is that currently, in_ncsf is using the method equivalent to "w/o Sloped Edges (current)" on that page. The calculation of the data points at the start and end of the data is only affected on any interpolation that uses 4 or 6 points in it's calculations. The 2-point interpolations are unaffected by this. Another thing to note is that using the slope to calculate those points causes every interpolation method to have the start and end go through the original start and end data points. This doesn't change the data in the middle, though.
What I'd like to know from you all is which interpolations you'd prefer I keep (and maybe which ones I should add as well). I would also like to know which of the edge handlings I should use. As it stands, I am thinking I will remove the 2-point 3rd-order Optimal completely and possibly add in both the 2nd-order Osculating interpolations, and for edge handling probably use the slope or the nearest point as it seems to cause the data near the edges to be slightly more in line with the curvature of the sound.
Wow, that's a very informative collection of information there. Some of those look like they would sound horribly wrong (looking at you, 6-point, 4th-order Optimal 4x), so of course I'm curious to hear them. :P It also shows that higher numbers don't always mean better.
Seems there's a high amount of redundancy between methods though. Slight differences in wave shapes make even slighter differences in sound. I tested the 4-point, 3rd-order Lagrange against the 6-point, 5th-order Lagrange, and the differences are on average around 40 dB below the rest of the music, a.k.a. virtually inaudible (this test may not actually be completely accurate, but if it's not it's inaccurate in a way that would exaggerate the difference, not underrate it; it may be adding together two slightly different things that don't cancel out, thereby increasing the total volume). There's actually even slightly less difference between 6-point, 5th-order Lagrange and Hermite at the same. So, while more choice is often better, many of these are hardly even different, factually speaking, not as a matter of taste.
But speaking of taste, though some of them don't look like they come close to hitting the original sample points and you'd think that would be a bad thing, it doesn't mean they sound bad. It would be unfortunate if 4-point, 3rd-order B-spline and 4-point, 4rth-order Optimal were both gone due to their apparent inaccuracy because I think they give a good balance between treble and noise reduction.
So I think you could trim some of the fat. Like you said, 2-point, 3rd-order Optimal 16x looks almost exactly like Linear in the graph and they sound the same to me. Hermite and Lagrange are so similar that they're essentially interchangeable, though Hermite looks a little smoother and 2nd-order Osculating a little smoother than that. The 4 and 6-point variations of Osculating seem so similar that only one is necessary, 6-point being the smoother one. Honestly, I wouldn't be opposed to eliminating Optimal in favor of B-spline.
If it were my plugin, I'd be torn between, on one hand, including the best 4 and 6-point versions of each type for the sake of a compromise between completeness and reducing redundancy, and on the other hand, condensing it only to the choices that sound notably different but not outright terrible, in which case I'd go with Linear; Cosine; 4-point, 3rd-order B-spline; 6-point, 5th-order B-spline; and 6-point, 5th-order 2nd-order Osculating.
I don't know which treatment I'd give to the edges without hearing the last one you haven't used, but either of the two other options seems better than the current one. On a related note, I'm not so sure it was the edge handling change that caused the clicking I found. I just checked with linear interpolation on the old and new versions of the plugin, which should be the same and not be doing anything different with the edges, but the new one has the clicking and the old does not. And I take back what I said in my last post that it could have been caused by the change in oversampling because it happens with the other interpolation types even though there was no change for them. Don't know what is causing it.
I decided to modify the program I created to generate those plots to also generate the wave files of each interpolation. Basically, the two that look messed up (4-point, 2nd-order Optimal and 6-point, 4th-order Optimal) sound like they get very noisy (background noise) as the oversampling rate is raised. It's really hard to tell the difference on such a small sample, though, but those two stood out.
I like the suggestion you made to which interpolations to keep, so I'll do just that. Sure, more choices would be nice, but as you said, a lot of these do create waveforms that look very similar to each other, so it would help reduce the amount of code and possibly cause a slight shrink in the final DLL's size. I'll also go with using the nearest point instead of the current point. I'd use slope, but thinking about it, it may cause some slight issues with the SWAV if they have to pull data from within their very last sample (slight possibility of them overpeaking). I'll have the update out within the next hour, but to be safe, check back at Noon EDT and it should be uploaded.
CyberBotX, sorry for being a bit OT, but is there any chance for you to release your GSF and 2SF plugins as well?
pmh will be hitting Game Boy Advance games soonish, and neither CaitSith2's plugin (extremely old sound emulation core) nor viogsf (uses VBA-M, so sounds better, but has some issues IIRC) are optimal choices.
As for 2SFs, I'm still on the fence what should I include on pmh. Those playback issues on Etrian Odyssey 3 are a little worrying, and some of Kirby SSU tracks I've checked sounded completely different (but then, I was using Optimal interpolation, and given how it distorted the first sample you posted, maybe that was the cause). It would be nice to have a matching 2SF plugin for easy comparison.
I can clean those plugins up and release them. Probably do that later today, if not, over the weekend for sure.
I might update the GSF plugin to the latest SVN revision of VBA-M. The only problems I really see with it is that GBA music played through it sound like they have extra background noise in them. I don't know if that is due to the quality of GBA music or if it's VBA-M at fault or if I need to mess around with the compiler/linker optimization settings. The last one seems like it'd be dumb to need, but I really don't know what the cause is. I know the Mother 3 soundtrack in particular sounds really background noisy, but kode54 posted recently saying he thinks it's the GSF set's fault, not VBA-M. I'm not sure what to think there. If there is a better GBA emulator out there to use instead of VBA-M, then I could look into that as well, of course that would delay me releasing a GSF plugin then.
As for the 2SF plugin, the main thing it would lack over the official in_2sf that is under the DeSmuME tree is the Sound View window, I believe it was called. But it does use DeSmuME 0.9.8 as it's core, so there is that. I also originally had it designed to be able to load a DeSmuME 0.9.7 save state if it existed in the reserved section, but the only time I ever used that was for a heavily hacked Phoenix Wright 3 2SF set, and the plugin can't load any 2SFs that use the old Yoshi's Island driver otherwise. The version of the PW3 set I have was a pain in the ass to make and I don't even think I have things set up to let me make it again, so I'd rather not continue including such hacky code in the plugin. Plus with me wanting to move over to NCSF, the whole problem with PW3 is a bit of a moot point to me now.
*ahemmmm* A...bout the GBA... It's not the GBA set's fault, hell, it's not even the emulator. It's the system itself.
To keep the cost low, instead of a dedicated sound chip, they only put in two lo-fi PWM channels and called them a sound system. It was up to the programmers to squeeze any sound outta them via software, and they can get no better than 8-bit resolution in the final mix. That's where the 'background noise' is coming from.
TUK actually asked a good question in the end of this thread's creation. Enter MIDI+SF2 sets created via GBAMusRiper. Properly setting up your SF2-Compatible MIDI player of choice (I use XMPlay! Gotta love Sinc!), you can see EXACTLY what I mean. Starmen.Org even hosted a MOTHER 3 Ultimate Music Rip on the front page once, via this method!
...Figured I'd set the record straight, or something. ;)
Anyway, good on you for initiating the first step to try and better a method of DS Sound playback, the way that dude tried for the GBA. Only this time, it'll truly be system-wide, not dependent on a lone sound engine! :D
BTW, thanks for setting up that Interpolation Illustration page, now I KNOW 'Optimal' was missing the sample points nigh-entirely! I've currently chosen 6-Point Osculation as the ultimate compromise between Hi-Fi and Ridding the Tinniness. MEGA KUDOS! XD
I might as well just give you this now even though I'm not quite done with it yet. Mystery Dungeon - Shiren the Wanderer [Fushigi no Dungeon - Fuurai no Shiren DS] (2006-12-14)(Chunsoft)(Sega) (The leftover stuff at the end is post-game bonus content I haven't played yet to be able to organize. Also, there should be an extra ME Collection track, "Event Item", but it must be in with the sound effects because it's not extracted with NDStoNCSF.)
I appreciate your adoption of my suggestions. :) Of course, I've found another issue though. :P In this and the previous version of the plugin, there's an odd regression. "Puzzle", number 8 in the set, is a particularly good example to work with to hear this. With the newer versions you can hear extra noisiness (sounds like harmonic distortion), but it's not there with the revision from the 7th. Track 9 was the one I found the clicking on before, which happens about 1:25 into it, and it's still there with the latest version, so indeed it seems the edge handling was not the cause. Any idea what could be doing this? What changed besides interpolation and edge handling since the version where you fixed the volume?
Sorry it took me a bit of time to get back to this. I was busy yesterday and got sidetracked today. Anyways...
So I played track 9 and seeked to just before 1:25, and maybe it's my laptop's speakers but I didn't hear the clicking. I tried with each of the interpolations to make sure. The only thing I can really tell is just a little after 1:25, there is something that sounds like it could be overpeaking, but I'm not sure and only seemed to happen with no interpolation or Cosine interpolation. The revision on the 7th (v1.2) is when I did the volume change. Since then, I made the interpolation changes as well as the start/end handling change, and just some minor optimizations that shouldn't have affected playback. So at the moment, I'm a bit stumped as to what is going on.
As an aside, I will need to talk to fincs about FSS. I found that some tracks from Knights in the Nightmare are sounding incorrect compared to their 2SF counterparts. Specifically, listening to the beginning of track 2.01 - Battle in the Abandoned Church, there is a cymbal in the background which sounds normal in the 2SF but sounds cut off in the NCSF. I confirmed that it also plays incorrectly on the DS itself via FSS, so it's not an issue with my end of the plugin.
Do you have some fairly decent headphones or speakers to plug into? I expect it would be tough to hear on laptop speakers. It actually doesn't come anywhere near to clipping so it's not that. Could you hear the extra noisiness on track 8?
Maybe just going back to v1.2 and updating the interpolation options to the current ones would solve the bug? I have no other ideas. I wish you could hear it so you could pinpoint the cause. As my volume-boosted difference file hopefully showed, I'm not just having an auditory hallucination! :D
A minor update since I've been quiet the last few days (and this is because I was trying to clean up DeSmuME's code for the 2SF plugin and it sucks doing that). kode54 had created a foobar2000 plugin based off of my Winamp one. He also informed me that interpolation needs to be using a circular buffer, which he then implemented into his foobar plugin and showed me the changes he made. I put those into the Winamp plugin, but then I noticed a lot more popping occurring, including being able to hear pops in the the song that nothingtosay mentioned. He is looking into this and will get back to me, but I will wait to release an update to the Winamp plugin until then.
And now the plugin has been updated. I again want to thank kode54 for providing the fix for this. I don't know if it'll fix the playback issues completely, but I believe it should help quite a bit.
. . . Either your Interpolation methods are mislabeled... Or the mixing methods are REALLY losing their power. They're WAY tinnier now than the last version. I just wish Ian at Un4Seen would let us use his Sinc method... :/
I'll talk to kode54 about it. We'll either find a better fix for this or I'll just revert the Winamp version back to how I had it, which would unfortunately cause the issues that nothingtosay reported to come back.
Found another song that's not played correctly vs the 2SF rip.
From Etrian Odyssey 3 set, track 26 Labyrinth VI - The Vengeful God in the Dark Ocean Abyss [000e - DUNGEON_6]. Roughly 2 minutes in, one of the instruments is missing in the NCSF rip.
I haven't looked heavily into why there is an instrument missing in that song. I did try to see if it was because of the track allocation returning a -1, which only happens if the patch it out of range or the instrument is out of range (or if the channel can't be allocated), but that wasn't the case. My only other guess at the moment might be due to an issue in FSS, but I didn't check the song on the DS itself to see.
On another note regarding FSS, I did bring up the issue with Knights in the Nightmare to fincs, and he thinks that is because of an ADSR problem. He had yet to look into fixing that, but once he does, I will update my version of as well.
As an aside, partially due to talk of the interpolation routine possibly needing work, I did some more research and I believe I managed to get Lanczos interpolation implemented into the plugin. Lanczos uses the sinc function, so it should be close to what is being looked for as far as Sinc interpolation goes. There is a new build of the Winamp plugin up that contains this.
Alright, looks like you've got it! The clicks have disappeared on Shiren track 9 and so has the noisiness on track 8. I've only tried out those so far, but if I encounter any other issues I'll let you know. Good work.
Well, kode54 has been helping me improve on the interpolation routine, including fixing the Lanczos one so it gracefully handles downsampling (basically if the SWAV is playing faster than the output sample rate). As an aside, though, I am wondering if I should add (or replace Lanczos with) Blackman-Harris sinc interpolation. It would basically still be using the sinc function, but it would be windowed using a Blackman-Harris window instead of a Lanczos window.
I'm still waiting to see if fincs will do any updates on FSS so I can update the plugin for that. Otherwise, I shall be putting up a new version here shortly that contains the fixes to the Lanczos interpolation. It also contains a lot of code cleanup (something more that I care about) and also makes it so the NCSF plugin will use 32-bit samples up until it is about to return the buffer to Winamp, at which point it will then clamp the samples to 16-bit. This hasn't really given much improvement from what I've noticed, but it might be somewhat helpful if a song possibly starts clipping before it gets to the ReplayGain section of the code.
Dude, three words: You fixed it! Even those stubborn samples I mighta mentioned on IRC Wed. Nite (or was it Tues?) are no match for the Sinc method in use at the time or writing this post!
However, based on the Interpolation Comparison page, if you must include Blackman-Harris, I'd suggest that both it and Lanczos be width 8, for detail's sake. ;)
Well, the Lanczos one in there at the moment is 8 width, and it uses 16 points for interpolation (the only reason I didn't label it as such in the config dialog is because it can be modified in the code, the number of points used would be twice the width of the window). Blackman-Harris would just be a different window on the sinc function. There are also plenty other windowing functions I could use, but since I have a hard time hearing the difference myself, I'm not sure which ones would sound better than others. Just from looking at window functions on Wikipedia, I can only guess that the ones with higher equivalent noise bandwidth (B in the labels under the graphs) are better, but I believe it would also depend on the Fourier transform of each as well.
As for the interpolation comparison page, it's possible that I am not calculating things correctly on that page for widths lower than 8. I did that prior to the latest code changes that kode54 provided.
Also, would it be too much to update the comparison page those few times you try to reshape the aural landscape? ;)
EDIT: This just in: This interpolation method is so powerful, that with my current processor (Pentium D 830), I had to bump the playback rate down from 192KHz to 96KHz. And that's from trying to play the final boss themes from my personal rip of Sonic Colors! (Coming Soon to NCSF. We'll talk. ;))
Just put out a minor update, kode54 mentioned that the phase offset needed to be scaled as well.
If anyone wants, I could put out a separate version of the plugin containing multiple sinc interpolations with the various windowing functions, and you guys could tell me which one(s) you'd like me to keep in the plugin. kode54 said that the Blackman-Harris window sounded like it had aliasing. I can never hear the differences between the interpolations (although I can hear a difference between any and no interpolation). :(
OK, I have a version up here: in_ncsf_sinc.zip It contains the sinc function windowed to 16 different windowing functions (all of them from that Wikipedia page I linked earlier). Rectangular is pretty much just a straight sinc curve, Lanczos is what is currently in the official plugin. As far as I can see from graphing the sinc curves for each of these windows, most of them will probably sound similar but have subtle differences in them. Let me know which one(s) you like the best.
Okay... I tried the experimental plugin, and I only have one complaint: Is it optimized?
I've been using the Mega Man ZX games as a benchmark, seeing as they had the majority of those stubborn samples I mentioned not too long ago. When I came to the following tracks:
Doomsday Device (MMZX) Black Burn (MMZX)
The plugin stuttered slightly, and one of the cores of my Pentium D was in FULL USE. (50% overall CPU usage) That was using 96KHz playback, BTW. All 16 methods suffered from said slowdown, EVEN Lanczos, and it never slowed down in the regular plugin!
Anyways, I spotted two methods also suffering from aliasing, namely Rectangular and Poisson. So, I guess, feel free to eliminate those two. Or not, it's your call. ;)
I'm not planning on implementing all 16 methods, maybe just 1 or 2 of them, depending on what others say, unless others really want more than 2 sinc methods for some reason. Plus the experimental plugin might not be as optimized since it is made in a way to allow all 16 sinc methods to share code which could cause a slight bottleneck (since it has to calculate which interpolation was chosen every single time it goes to interpolate, which can be costly I suppose, but I wasn't worried because it was a test plugin), plus it uses a lot of memory for holding the lookup tables in memory. Tha overhead won't exist in the real plugin because it will only need to hold 1 or 2 sinc lookup table(s).
As for the aliasing problem, I've tried looking up what would need to be done for that, and I admit to being a bit mindboggled in regards to the solutions. A lot of them mention things like low-pass filters or anti-alias filters, but I couldn't find anything on how to implement that, and I also want to keep things simple. As far as I can tell, supposedly a proper sinc curve handles anti-aliasing already, but I'm not 100% sure on that either. As you mentioned, the Rectangular and Poisson windows do not seem to help with anti-aliasing.
Just be glad it's only two out of the 16 that are the culprit. ;)
Also, help might come my way in the form of a processor upgrade... I just hope Winamp and/or XMplay's plugin system can properly utilize hyper-threading...
Oh, and just to help visualize things, I updated the Interpolation Comparison page again, I removed the previous plots of the Blackman-Harris and Lanczos interpolations, and show all 16 of the windowed sinc interpolations. It does seem to show really well why the Rectangular and Poisson windows don't sound right. Their plots just look off compared to all the other windowing functions.
I'm still gonna wait to see if anyone else has input on those windowed sinc interpolations, but as it stands, JFD has said only Rectangular and Poisson don't cut for him (due to aliasing), kode54 has told me that Blackman-Harris also produces aliasing, and soneek says that Hann-Poisson, Lanczos, and Blackman sound great to him. That probably means I might use all 3 of those in the final version of this, but we'll see. I think I'd rather just limit it to 1 or 2 of them. If I do, I believe I'd keep Hann-Poisson from soneek's list, and if I do 2 of them I'd also keep Blackman. I wouldn't keep Lanczos if only because it has a lower noise equivalent bandwidth compared to the other 2 mentioned.
As an aside, I have put the NCSF/SDAT tools onto GitHub. I am going to wait until I put out another update to in_ncsf before I put that on GitHub as well, mainly because I need to refactor the directory structure before I upload it to there.
Okay, I did a tournament-style comparison of all 16 and my result is that 12 of them are virtually identical. Some are closer to each other than others, but never to an audible degree to me, with my speakers turned up to a decent level. The only ones that were clearly inferior were Rectangular and Poisson, with strong aliasing, and Welch and Hann-Poisson, with slighter aliasing mainly noticeable from not being present in the others. I haven't experienced audible aliasing from Blackman-Harris with the two tracks I've demoed, but of course, that is only two tracks (I don't have forever and this is time-consuming!).
My opinion is that if you picked at random any of the others to keep you'd be fine.
I've been doing a little more research and I see a lot of things that say the Hann window is best for when you are unsure what kind of data you have, and considering how the data we're dealing with could be anything in nature, I believe that the Hann window is the way to go. I will update the plugin so it contains only the Hann window for Sinc interpolation.
Also, fincs finally checked out the issue I had with the ADSR of Knights in the Nightmare and concluded that it was using invalid values for the decay (it has a 0xFF which isn't technically valid). I'm actually doing some checking to see if I can find any other music that ever runs into similar issues, but that might take some time. As a result, I will not have the update for the plugin quickly.
As for Knurek's issue with EO3, I'm not sure if it's just me or not, but I cannot hear the missing instrument in the song (it also doesn't help that I have to wait until it gets to the 2 minute mark). Something that would be more helpful is if there was a song with an issue very early on, like in the first few seconds. At first I had thought that maybe it was also an ADSR problem, but that song doesn't hit the code which checks for an invalid ADSR value, so it's not that.
Also, I am going to have to speak with kode54 about the interpolation buffer he added. It seems that some songs start overpeaking or popping or something on ANY interpolation if there is no ReplayGain on the tracks, which in essence is going to cause the calculation of ReplayGain to be flawed as well. Strangely, it only happens when interpolation of any sort if used, and doesn't happen if I turn off interpolation.
0:03-0:04 in the sample I posted, that's where the missing instrument starts. One of the (background, granted) sections of the song seems to fade out very quickly, compared to the 2SF rip.
Anyone else not hearing that?
Looking at the SWAR files, either sample 0E.swav or 0B.swav are the culprits here. Hope this helps.
This is a very nice alternative I must say :) So will there be options to mute instruments or something like that (already was mentioned for Rune Factory 3 water noise)? I rememember I used this vio2sf menu to find and mute the damn water before dumping the track to wav screenie
Edit - another small thing: Made a test for Summon Night: Twin Age (corresponds to [Summon Night Twin Age - Seireitachi no Kyoumei] (2007-08-30)(Flight-Plan)(Banpresto).7z from joshw) and I notice the drum thingy instruments are a lot quieter in NCSF rip in these tracks: 0002 - bgm102_dungeon03.minincsf (Strength of the Sword from 2sf rip) 0003 - bgm120_Boss.minincsf (Boss Jam) and maybe on one or two track more. Don't know what the volume for those should be on the real NDS (don't own it anymore). Here's a NCSF rip: http://ryushare.com/hmrskqsjz9ev/SNTA_NCSF.rar
Knurek, the samples you posted are all well and good, but I need to be able to hear it in the actual plugins (both the 2SF and NCSF ones), and having to wait for 2 minutes of song is a bit much to try hearing differences. The main reason I was able to get fincs to fix the issue with that one Knights in the Nightmare track is because it happens right in the beginning and was easy to check back and forth between the 2 sets. Plus, there is a possibility that the issue may have been fixed as some point, so hearing it in the plugins is better than using the samples provided.
cooljacker, this is the first time I'm even seeing that window. :o I wonder how recently that got added to vio2sf. I know that the official vio2sf that is in the DeSmuME tree has a Sound View window of some sort, but my own 2SF plugin doesn't have that (I never took the time to port that code over). As it stands, I may have to make a modification of the NCSF specification to allow for changes to the SDAT.
As for the volume, I believe NCSF might possibly be quieter than 2SF, if only because of the utilization of the volume from the info block of the SDAT for the SSEQ. I could check sometime to see how it sounds on the actual DS. As an aside, I technically have the NCSF rip for Summon Night: Twin Age already, but it's untagged. I might actually modify the NCSF listing page to include both tagged and untagged sets, since I do have a few other sets that I ripped but haven't tagged yet. (Note: Even if there is a 2SF set for any of them, I'd still leave them untagged right now because I've got other things going on.)
Thank you very much for replying, CyberBotX! About the vio2sf, it was actually unofficial version compiled from desmume repositories and it was done by a kind person from here but I forgot who it actually was :( I keep the version for Winamp 5.x (both files go to plugin directory), could be useful so here it is anyway http://ryushare.com/9i6i6vnttoda/viowmp5.rar
Alright, so it took me some time for testing and all that, but I have uploaded a new version of in_ncsf just now. On top of it replacing the Lanczos window with the Hann window for the Sinc interpolation, I also made my own version of a ring buffer to handle the SWAVs in a bit nicer of a fashion (and it also doesn't have sample lag at all). I still have to give big thanks to kode54 for his original implementation, as if it wasn't for him, I never would've realized that I even needed a ring buffer in the first place. One other thing I should point out is that I apparently did the clamping completely wrong (16-bit integers do NOT go between -0x7FFF and 0x8000, they go between -0x8000 and 0x7FFF) which is what was causing the clipping-like sounds (I lied, it happened even with no interpolation, and was an error on my part). This has been fixed now, and I decided to use std::numeric_limits to make sure I can't get it wrong again.
I think now that I have the plugin in a bit of a better position, I will probably start working on the tools again. Before that, though, I am going to clean up my local source tree for in_ncsf and put it up on github.
cooljacker, do you happen to have the source of that vio2sf? The RAR only contains the compiled version, so I wouldn't be able to implement something like it into in_ncsf (or even my version of in_2sf) without the source code.
Ah, indeed it is. I can look into trying to integrate that into my plugins at some point. It's not really high up on my list, though. I really want to put the plugin to the side for now and work on the tools some more, they have been collecting dust because of all the plugin work.
Start of the sample I prepared, actually. Yes, it's two minutes in. CBX should still be able to wavewrite it though, should be much faster than listening through the whole two minutes for that part.
Alternatively, there's 0000 - OPENING.minincsf from the same game which has a glitch right at the beginning of the song.
You had me thinking I had misinterpreted Knurek, Franpa. In any case...
Knurek, I used Winamp's Format Converter to output .wav files for both the 2SF and NCSF of that song, both without interpolation, then I loaded the waves into Audacity. Here's something interesting. The first 0.00225 seconds of the song (I'm not sure how many samples that) are identical between the 2SF and NCSF. But then the 2SF has a period of silence up until about 0.00535 seconds in, which the NCSF does not have (it's actually playing something for those 0.003 seconds). At that point, the songs no longer sync up, and the 2SF encounters another moment of silence that the NCSF doesn't have for a few more milliseconds. It's hard to tell at that point, but it seems almost like the 2SF syncs for those first 2 millseconds and then it is behind by about 1 millsecond after the silence portion. There are some very slight differences in the data, but just from looking at the waveforms, they appear to be almost the same, with the NCSF maybe being a hair quieter. So as far as I can tell, the songs are playing nearly the same, aside from the 2SF's two silence gaps at the beginning.
Keep in mind that the above is from the latest version of the NCSF plugin (v1.8) and my 2SF plugin, which is based on a modified DeSmuME 0.9.9 (from SVN, the release only happened about a week ago but I haven't updated my own sources to the release and I think my version is probably close to the release's version anyways).
As an aside related to the above, I have put the NCSF plugin (as well as my xSF framework) onto GitHub, at https://github.com/CyberBotX/in_xsf. This also contains my versions of 2SF, GSF, and SNSF plugins. The 2SF plugin, as mentioned above, is based on DeSmuME 0.9.9, but it will NOT play 2SF rips that use save states (in other words, it only plays 2SFs that use Caitsith2's Legacy of Ys driver). The GSF plugin is based on VBA-M and uses a pretty recent revision of it (it's probably from about 2-3 weeks ago). The SNSF plugin is based on snes9x 1.53, but I also added a few additional resamplers based on the interpolation I added into the NCSF plugin.
I'm pretty sure the delay in 2SF is caused by the fact that the plugin is emulating the whole system, so it needs to boot up system, set up the music driver, etc. I imagine that NCSF, being high level, does not need to do that.
0.00225 seconds should be roughly 100 samples, given 44100 Hz frequency.
//EDIT
Sorry if this is a stupid question, but is there any way to download plugin builds from that github page? I can only find links to source files, and I don't have VC installed on my machine.
Well, that's the thing, the way both plugins are set up, since I did use some concepts from the vio*sf plugins, is that silence at the beginning is skipped. If the two weren't identical for those roughly 100 samples, then I could see a startup or something causing the delay. But the fact that after 2 milliseconds the 2SF becomes silent for 3 milliseconds, plays for about 2-3 milliseconds, then goes silent again for another 3 milliseconds before continuing, is a bit strange. This is probably also why the output of a single channel from both in_2sf and in_ncsf are not the same.
Now, I'm not saying that it's not possible for the NCSFs to be wrong. But I am saying that is possible that the 2SFs could be wrong (whether they are or not is another question). They could be wrong either because of DeSmuME or because of the Legacy of Ys driver. And NCSFs could be wrong due to FeOS Sound System.
FSS is missing some commands. Although he helped me fix the recent ADSR problem, fincs has given no word on when he plans on looking into the disassembly of those unimplemented commands. There has been at least 1 or 2 other people in #dsdev on Blitzed who said they would try looking into it, but I think the problem is that no one else has the disassembly of Nintendo's player for the Nitro Composer format. I personally do not even know how to pull a disassembly from a DS ROM, and even if I did, I do not know ARM assembly to interpret the results.
ALSO EDIT (Because I was typing my post before your edit): No, but I could put up the DLLs. Assuming that in_ncsf is already installed, you would already have zlib1.dll in place, so you would just need to drop the DLLs in and you'd be good. I'll post links to the other 3 plugins after I compile Release versions of them.
One thing I've noticed with the GSF plugin though, and not something I've found a fix for: It seems that if it plays a GB or GBC song (that have been converted to GBA format) and it plays for too long, the plugin as a whole enters a state where it refuses to play anything else at all. Restarting Winamp fixes it, obviously, but I am unsure why this is.
Thank you, will play with them when I get back from work.
Not sure about the GBS2GSF crashes, I seem to recall something like that in the original GSF plugin as well, I think the last version fixed that. From CaitSith2's changelog: Permenantly Disabled SWI 3 (stop), as there is no pratical use for it in GSFs.
Hmm, funny thing, that Etrian Odyssey 3 track I mentioned that has a glitch in the beginning (01 That's the Beginning of the Adventure or 0000 - OPENING)...
It has the same glitch when playing the 2sf rip with your plugin. Nothing like this is happening with vio2sf... Not sure if this helps you in any way, I guess there is something your resampler framework does that's not really compatible with this track.
The other problematic track from the set plays fine, just like vio2sf.
I couldn't help but notice that the 2sf plugin doesn't have any of the new interpolation options. I hope you'll be able to add them as well.
When I did my test with Audacity, I had interpolation turned off, so I don't think the interpolation is part of the problem. I know that vio2sf is currently a crazy mix of old DeSmuME 0.8.x code and some code from the 0.9.x branch. My in_2sf is entirely DeSmuME 0.9.9, though. (Also, even though it says it's from SVN, it appears that nothing has changed since the revision I used, other than Mac OS X port stuff.)
As for the 2SF plugin lacking the new interpolation options, I would probably have to do some heavy hacking of DeSmuME's SPU to handle that, mainly because they wrote their interpolation under the assumption of there only being 2 points, plus on top of that, they technically aren't handling looping SWAVs. I'm also a lot less familiar with some of their SPU code. There were only a few parts of it that I was able to utilize in in_ncsf.
EDIT: Actually, I may be wrong about them not handling looping SWAVs. I haven't really looked over the code heavily enough.
This looks like the EXACT thing i need right now! I spent the last three or so days researching ways to play NDS music and maybe dump it to .wav, specifically the music in Time Hollow. It was frustrating.
I experimented with various things, like the ever-crashing VGMTrans, which does have a nice palyback function. I'd have to clunkily record my PC's StereoMix though.
I then tried various methods to transform the SSEQ files into MIDI and tracker formats, and they all either played completely wrong or, at best, sounded slightly off, since none of them properly supported ADSR.
I even played with the idea to use a NDS music player that embeds the SDAT in a Rom to play with desmume, but... i'd need to get the development environment for th DS / ARM first. What a pain.
Now, i'm not sure how exactly this works. This is a Winamp plugin that plays what sdat or sseq/sbnk i feed it?