vgmstream equivalent for sequenced files by jurassicPieter at 6:23 AM EDT on August 4, 2009
I'm thinking of working on an equivalent of vgmstream for sequenced music. First reason I want this is that my winamp plugin map is a mess. Often the file formats are associated with multiple input plugins. Second, it's easy to make a container format that contains a soundfont(or reference to it) and a midi. There's extended midi with soundfonts added to it, but it's very badly supported to make them. Third, it might be a good start to figure out unknown sequenced formats, like many gamecube titles. Or embed a stream and mix it with sequences, like they did with Super Mario Galaxy.
The only difference with vgmstream is that it needs a custom container file format instead of playing the originals.
my idea was to make a container file format that works with the typical chunks structure.
for example a midi with soundfont would look like this:
MAIN chunk size of file player number: 0x0001 = midi with embedded soundfont --IDV3 chunk --size of tagging info --tagging info, like artist, looping, etc. --MIDI chunk --size of embedded midi-header --midi-file --DLS chunk --size of embedded dls-header
the player number is used to get the right playing information: 0x0000: midi with general midi soundfont gm.dls in windows\system32\drivers\ 0x0001: midi with embedded dls-header 0x0002: midi with external dls-file 0x0003: play midi file with 8-bit sounds 0x0004: sseq with embedded sbnk/ssar-file(s) 0x0005: sseq with external sbnk/ssar-file(s) 0x0006: IT/XM/S3M etc.
and what extension and name should this plugin have?
suggestions are welcome. If people find it a good idea i plan starting making a skeleton code that should make it easy to write a player that just has to send sequence data and soundbank information.
Sounds like an interesting project, if a bit ambitious. I know Destop did something like this for N64 music before USF came into being, though just targeting the main CSeq system. One issue (that you might encounter elsewhere) is that MIDI or MIDI-esque formats have a lot of potential for custom controls. I know this is one of the things I encountered when trying to do generic USF ripping, stuff like custom vibrato controllers (iirc) were more common than I'd have expected.
For the file extension, the first thought that comes to my mind is *.GENS for "GENeric Sequence" (since the first thing I thought of when you mentioned a container format was GENH), but that might be confusing to people who use the Genesis emulator GENS.... Perhaps *.SEQC for SEQuence Container?
It's an interesting idea you've got, but I have to agree with HCS - this might be rather difficult to achieve. Good luck. I don't have any additional input, but if I can help, I'd be happy to. Mouser X over and out.
thank's for the ideas. I brainstormed a little bit about it. The biggest problem so far is indeed custom effects(could be covered with scripts or is that too slow for dsp?) and the conversion from instrument+soundbank to a stream format that winamp plays.
I have some programming experience with playing sequences with the fmod library, but fmod is not capable of sending individual midi messages, just a matching midi and soundbank support(and with file callbacks i could read data from this generic sequence container format). And i wrote a lua script version for generative music working on the old-fashioned sleep and midiOutMsg functions. Sadly it was hard to make good generative music so i haven't done anything with it anymore.
I don't have experience with winamp plugin programming yet.
Originally vgm format and adplug were also meant for emulating every format if i recall.
As for lua: when i was thinking about it you could make dynamic streams played. Like picking the right looped music like they used in the halo games or make Conker's Sloprano dynamic(you could even make a small UI to 'control' it).
Re: by jurassicPieter at 6:27 PM EDT on September 3, 2009
well i haven't got much feedback from any of you, so it was a bit on low priority. I tried to look at libraries that convert midi-data to an audio stream to start with and i think softsynth is easiest to integrate with winamp and i have a skeleton version so far that should play a 'scf-file' that uses a chunk-based format and ignores unknown chunks(similar like the 3ds-format chunk structure). Now all i need is a simple midi player container file that uses gm.dls which was the next step to try.
As for my idea of lua to make dynamic streams playable: it's hard to integrate in vgmstream and i have no idea how to make seeking work, especially because of the way how lua-script code is executed.
I've been working on it to also test RSEQ's generated and the start of sequenceStream is made:
- file extension is sqf. File contents is just a lua script. - midi rendering is done with fluidsynth. timidity seems to only have documentation to use it as a standalone app, so fluidsynth was my choice. - the lib contains of 3 types of Classes: FileReaders, Sequencers and Soundbanks - FileReaders is what you sent to sequencers and soundbanks. FileReaders have all reading functions including variable length reading and chunk reading. So far there's a complete file reader, a subfile reader and a memory file reader. - sequencers contain the decoding a file to midi events that are sent to the system. So far my experimental RSEQ(inside brsar files), MIDI and SSEQ(taken from sseq2midi) are working. - soundbank support is sf2 only which is standard soundfont support in Fluidsynth. RBNK, SBNK and DLS are next - i made batch files that generate sqf files for brsars.
I made a first version that plays, but I'm rewriting it so it's better written and easier to edit.
added next couple of things General: linked with boost functions instead of stl gave me a big speed boost. get info and title are linked to a lua script function, so you can add tagging support. adding comments so people can help with adding file support in the future. looping support working, though still experimental
FileReaders: zipped files: I talk about gz, not zip. Reduces file size, and seeking works even though it's slow. files in archived files: Now it expects the archive to be tar, but just one virtual function needs to be overwritten for different types or archives.
Soundfonts: Pokemon Box wt+pcm format: probably the first released set once i release the plugin since the game uses midi and a simple wt+pcm format.
If you use something like SF2 for the sample bank, then you can use a library like FluidSynth to play the MIDI files with the custom banks, regardless of the playback hardware present in the host system.
Ehm, i'm using FluidSynth. However the documentation is very vague when it comes to making your own sample banks. I figured it out though. I still had to implement my own soundfont system though. Most of the time i keep refactoring to get a good starting point to work with.
Might make a sourceforge page soon so people can work on it, but i want a good basis to start with.
XM files play now too and you can even play the sequences of the XM file with a different soundfont or viceversa. It sounds silly!
One thing i missed in most winamp plugins is the lack of looped playing individual songs. Now it's relatively easy to do. I made all Doom songs looping. Nothing beats playing 'At Doom's Gate' for infinite while using a SB16 synth soundfont :)
I have a sourceforge project now here: http://sourceforge.net/projects/siiqstream/
it's still a bit buggy, since there are so many parameters in soundfonts and i've been fiddling a lot with the annoying linking options of visual C++, packing alignment and other stupid things of (Visual) C++.
I restarted for the sourceforge project to add file support as easily as possible. It's now a matter of using inheritance and overloading a readFile and play function. And you just use things like addEvent and addSample, etc.
Length is calculated and looping is implemented too. Seeking should be done soon.
Can people with Visual C++ test out if the sourceforge version compiles on their computer? It's written for visual C++ express 2008.
i still have to figure out why i get a heap corruption error from winamp. Seems like in_sqf uses different heaps?
compiled DLL version: I upload it tomorrow, need to go to bed now
Some test games Doom [DOS] (1993)[MID SF2]: http://www.megaupload.com/?d=RLLXTUBJ Wacky Wheels [DOS] (1994)[MID DLS]: http://www.megaupload.com/?d=JIM9A80X
hcs, if you want to compile it on a different platform: there are several external libs i used: Lua Tolua++ Fluidsynth Zlib
After adding .\siiqstream\external\VC2008\include\lua to my include paths, the sequence_lib project successfully builds which allows the plugin to link. The sqf_play project has a linker error:
MSVCRT.lib(crtexe.obj) : error LNK2001: unresolved external symbol _main
For some reason it can't find the main.
The plugin crashes both XMPlay and Winamp when loading track times for Doom. Winamp gives some errors:
The linking error is known and fixed now. It seems all the compiled libs used different settings and heaps, so i removed the libs and added the source code. Now you don't need external dll's in winamp anymore too.
here's my compiled version: http://www.megaupload.com/?d=GCZDXEWN there still seems to be some weird thing with loading lengths. It runs perfectly when running from the Visual C++ IDE, but it crashes if i start it outside the Visual C++ IDE
Here are some random outputs converted to OGG with the filewriter winamp output plugin and using Siiqstream to read the SQF files. I shouldn't add more format support as long the winamp plugin crashes if you don't start it from the visual studio IDE. At least i got rid of most heap corruption errors and it runs more stable.
Doom with Siiqstream: http://www.megaupload.com/?d=2N01T6O0
Prince of Persia (1983 game) with Siiqstream: http://www.megaupload.com/?d=K0ROL5BI
doom sounds pretty nice :) manakoAT says it works in winamp for him, but trying to run another file in the playlist doesn't (or rather, that's the jist of what I got from him) and the plugin consequently crashes. don't know if that helps at all
It seems it's a compiler setting that makes it crash, since the current version doesn't crash in debug and it does crash in optimized release version in fluidsynth code. I have made changes to disable loading soundfonts when it only needs lengths and also it doesn't create a fluidsynth synth and now the release version doesn't crash until it plays the next track(like manakoAT is experiencing).
In debug everything works, but if i read the output when all debugging code is verbose i see it allocates the memory with siiqstream memory unit, but that it deallocates with WinAmp's heap. To figure out this bug I need to read either the assembler or has to go through all compiler settings.
Next release will be adding RSEQ support, including my batch-script that makes SQF files of a brsar-file(but uses a sf2 for the soundfont so far instead of the RBNK and RWAR files inside the brsar).
by nensondubois at 9:10 AM EST on November 9, 2009
What about .arc files after they get extracted into multiple files somehow?
by arbingordon at 10:14 AM EST on November 9, 2009
one thing at a time, nenson.
though pikmin/luigi's mansion rips would be awesome :)
Great work, jurassicPieter! I'm looking forward to further development. :)
by arbingordon at 6:20 PM EST on November 16, 2009
here's some crap regarding pikmin/luigi's mansion, stuff the pikmin one has some windows dlls i found in the iso, really weird, even has the MZ header (I figure someone might be able to figure out the piki file format by reverse engineering plugPiki.dll) there's also some ILK files, no idea what they are, but they're tagged up as being made with windows when viewed in a hex editor, so i figure those also have something to do with the development
no idea on what to do with luigi's mansion
by unknownfile at 8:00 PM EST on November 16, 2009
from what i could remember, there are two versions:
- luigi's mansion / mario sunshine / pikmin. the driver is written in C and has all of the sequences scrambled together in a single ARC file. to get the offsets for the sequences themselves, look in JaiInit.aaf:
GoodNight.com in Luigi's Mansion is located at 0x00022CC0 in the file and is 0x00000280 bytes big
- zelda / mario kart. the driver is written in C++, and the sequences can be extracted out of the ARC file using common tools.
it's more of a script format than anything else, using all sorts of commands rather than simple key on/offs to do stuff.
edited 8:03 PM EST November 16, 2009
edited 8:10 PM EST November 16, 2009
by arbingordon at 5:19 PM EST on November 17, 2009
uf: any idea on what the dlls were meant to do?
by unknownfile at 6:30 PM EST on November 17, 2009
most likely debug shit left behind on the disk before shipping
The crashing by MegaByte at 11:28 PM EST on November 18, 2009
If it's crashing with a release build, but not the debug, that often means you've got a buffer overrun somewhere. It doesn't kill the program in debug mode since debug adds buffer areas around your memory allocations, but without the buffer, it crashes in release mode. Anyway, testing out a compile for myself...
Update: Okay, after fiddling with all the library settings, I got it to compile. The release build doesn't seem to even be recognized by Winamp. In the debug build, it was crashing whenever I loaded a Wacky Wheels file. But I could ignore the crash and continue execution through MSVC, and the file played. Doom played without crashing. Here's a stack trace for the crash:
msvcr90d.dll!_free_dbg_nolock(void * pUserData=0x160f60e0, int nBlockUse=1) Line 1371 + 0x3b bytes C++ msvcr90d.dll!_free_dbg(void * pUserData=0x160f60e0, int nBlockUse=1) Line 1258 + 0xd bytes C++ msvcr90d.dll!free(void * pUserData=0x160f60e0) Line 49 + 0xb bytes C++ in_sqf.dll!operator delete(void * pointer=0x160f60f0) Line 119 + 0xc bytes C++ in_sqf.dll!Seq_MID::`scalar deleting destructor'() + 0x3c bytes C++ in_sqf.dll!MainSystem::unloadMidi(Sequencer * a_Midi=0x160f60f0) Line 463 + 0x34 bytes C++ in_sqf.dll!tolua__MainSystem_unloadMidi00(lua_State * tolua_S=0x0721eb98) Line 819 C++ in_sqf.dll!luaD_precall(lua_State * L=0x0721eb98, lua_TValue * func=0x042993a8, int nresults=0) Line 319 + 0x16 bytes C in_sqf.dll!luaV_execute(lua_State * L=0x0721eb98, int nexeccalls=1) Line 587 + 0x14 bytes C in_sqf.dll!luaD_call(lua_State * L=0x0721eb98, lua_TValue * func=0x04299398, int nResults=0) Line 377 + 0xb bytes C in_sqf.dll!f_call(lua_State * L=0x0721eb98, void * ud=0x006777c8) Line 800 + 0x16 bytes C in_sqf.dll!luaD_rawrunprotected(lua_State * L=0x0721eb98, void (lua_State *, void *)* f=0x6c356b90, void * ud=0x006777c8) Line 118 + 0x1f bytes C in_sqf.dll!luaD_pcall(lua_State * L=0x0721eb98, void (lua_State *, void *)* func=0x6c356b90, void * u=0x006777c8, int old_top=48, int ef=0) Line 463 + 0x11 bytes C in_sqf.dll!lua_pcall(lua_State * L=0x0721eb98, int nargs=0, int nresults=0, int errfunc=0) Line 821 + 0x20 bytes C in_sqf.dll!ScriptSystem::runFunction(char * a_Script=0x6c3e631c) Line 29 + 0x12 bytes C++ in_sqf.dll!ScriptSystem::~ScriptSystem() Line 21 C++ in_sqf.dll!ScriptSystem::`scalar deleting destructor'() + 0x2b bytes C++ in_sqf.dll!MainSystem::closeAll() Line 60 + 0x37 bytes C++ in_sqf.dll!MainSystem::~MainSystem() Line 24 C++ in_sqf.dll!MainSystem::`scalar deleting destructor'() + 0x2b bytes C++ in_sqf.dll!getfileinfo(const char * filename=0x00677fec, char * title=0x006783fc, int * length_in_ms=0x0049c85c) Line 245 + 0x34 bytes C++ winamp.exe!0040f023()
Obviously, something is getting deleted that's not supposed to, called from the lua script. I added a line, if(a_Midi != NULL) in MainSystem.cpp before delete a_Midi; to try to get rid of the crash, but it seems that it's not a null pointer problem. I still get the heap corruption error. Given the cast you do in the lua glue code, could it be that it's not really a Sequencer object that you're passing for destruction? I'm not familiar with the code, so I'm not sure what to trace where.
Sorry for the lack of updates. A new girlfriend takes quite a lot of time. Not to mention my graduation project is time consuming.
From what i've noticed it seems to be using 2 heaps(a debug heap and a release heap) at the same time and some files are using the debug heap and some the release heap. It's weird since memory.h is used in every file. If I made the MainSystem g_Sys; object in the winamp plugin as MainSystem* g_Sys; i seem to solve at least one heap corruption. Another thing i figured out that i didn't make a few destructors virtual. I thought making the base destructor virtual was enough, but i ended up changing all destructors virtual. The amount of errors i get is dropping and stability is rising luckily.
The latest version crashes in release mode only if you don't debug it. If i attach it to winamp, it seems to crash on fluidsynth checking all voices. This is weird, since the synth is re-initialized if a file is played and no synth is created if it tries to get information from a different file.
Well, this is better. The release compile works for me now. You've got a mix of Multithreaded and Multithreaded DLL library settings. When I changed them all to be the same thing, I was able to remove the excluded library entry, which could be part of your problem.
Thx for telling. and sorry for lack of updates. I had some distracted side projects and got annoyed by the annoying heap corruption error. The project is not dead, it's just sleeping. New girlfriends can be time consuming(but pleasant:) )
MegaByte,Do you have the updated sln and vcproj file for me?
I was working on reading sdat archives first, then followed by reading brsar archives. The file formats are almost the same in structure with small changes and the endianness being different. All RSEQ files inside brsar's are like a mix of SSEQ and SSAR.
Also the newest version would make some functions for tagging available. Sadly it makes the older SQF files i provided not working anymore or without tags available.