Problem understanding the differences between streamed music and non-streamed midi music? by dj4uk6cjm at 7:06 PM EDT on July 28, 2016
I've been pondering for a long time about how exactly do composers utilize sequencers like the Roland SC-88 for streamed music in their soundtracks for video games and the same sequencers used to make midi style music for same said video games.

How do we know some soundtracks are streamed and some not when they use the same tools to produce both types of music? Shouldn't they all stay one type which is typically streamed? At least in todays modern video game music composing industry even though there are still rumors that midi music is used in these days as well.

What bothers me is why composers, whether they be the same composer (or a completely different composer) for the same game working on music for another game on the same console or handheld sometimes choose to make streamed based music for one game (lets say for example New Super Mario Bros. Wii) while some make midi music for another like Super Mario Galaxy 2 which are basically a year apart from release dates of each other.

It's okay if you guys don't want to comprehend what I'm talking about lol I don't even understand where I'm going with this at this point too but it seems to me that composers keep juggling between streamed and non-streamed music and if you look by the years some games have been released and history of video game music composer using streamed and non-stremed music in their tools it'll give you an idea.

Why are they doing this? It just seems odd when you put the pieces together.

Anyways I hope this generates a lot of discussion, nothing flammable or whatnot. Just curious why they're juggling between streamed and non-streamed music, maybe it's because they like trying new instruments in their songs and testing them out on streamed and midi? Who knows, we'll never know I guess.
by kode54 at 10:55 PM EDT on July 28, 2016
Streamed music typically implies that it is pre-rendered, pre-composed, and/or pre-recorded to some PCM or compressed PCM format, then read from the media at run time.

Non-streamed MIDI music would require the actual synthesizer, or samples from an actual synthesizer and some sort of mixer or hardware synthesizer chip to play. The sequence and sample data are read to memory at load time, and synthesized in real time.

The choice to use sequenced formats may be entirely up to space constraints, since streamed data requires considerable space on the media.
by hcs at 12:03 AM EDT on July 29, 2016
A point of clarification: A "stream" is technically an audio sample that isn't all in RAM at once; parts of it are "streamed" off of the disc or cartridge just in time for it to be played. kode54 is right in saying that what we deal with in vgmstream is really just prerendered or prerecorded. I'm going to call this offline audio, vs. online audio which is synthesized in real time (often called MIDI or sequenced).

---

There are a few reasons to prefer offline audio:

- With online synthesis every additional instrument and effect takes processing power, and even if you have hardware to do the synthesis the music is competing for hardware channels with sound effects.

- Instrument samples also take up precious RAM, unless you have a fast enough storage device to stream them. You may need to use fewer and lower-quality samples to run online.

- If you insist on online you may need to export the sequence data to a common sequence format to run on the target engine. Offline audio can be rendered and mastered once and will always sound the same on playback.

So offline, you can use any synthesizer, huge sample sets, unlimited simultaneous instruments, and any number of layered effects, and it will still use very little processing power in the game.

---

Downsides of offline:

- Size, for storage and transmission. Until fairly recently perceptual codecs like MP3 and Vorbis weren't fast enough to run alongside a game, so the best you could do was usually a 4-bit ADPCM, which still takes 1/4 as much space as an uncompressed PCM WAV (think of a CD taking 150 MB). Even highly compressed audio may be too much for a downloadable game, if there is a lot of music.

- It is harder to do any kind of dynamic effect. You might split a track into variants and store those all, but you can't easily turn off a channel, or change the tempo. Fully reactive or generative music may be impossible.

- As you can't usually fit offline tracks in RAM, you must stream them, so there is a question of storage bandwidth. In an open-world game, for instance, you don't want to have music competing for disc bandwidth as you're trying to load the next area.

---

There are a lot of games that have both offline and online music.

Consider Twilight Princess: cutscene audio is offline, most other audio is online. I think the offline audio was used in cutscenes for tighter timing: you always know exactly what frame every note will fall on when prerecorded, which a cutscene director can rely on. It may also be to free up memory for better character models and specialized animations.

F-Zero X is one of the few N64 games to use offline audio. Most N64 games use online, in part because the N64 has a very fast cartridge, with no seek time, so samples could be streamed without having to take up RAM. I understand that FZ-X went offline to free up processor time for high-performance graphics. Other games couldn't afford to use up all that cartridge space, and the FZ-X tracks are notably low quality as a compromise. There are a few sequenced tracks in the N64DD expansion kit, but they are in non-race situations (car and track editor).

---

All things being equal, I'd like to just do everything online, for the sake of flexibility. But if I can't spare the processing power, and I have enough space and bandwidth, I'll render offline.

It's the classic space-time tradeoff.

edited 12:15 AM EDT July 29, 2016

edited 12:56 PM EDT July 29, 2016
by dj4uk6cjm at 6:16 AM EDT on July 29, 2016
Umm...Wow! :O you guys said a mouth full lol thanks for your educated answers on the subject, I think I've learned quite a bit from this and I did have a theory where maybe why they do streamed and non-streamed music simultaneously is because of data restraints in the roms/isos. Well if this was one of the cases nintendo should've by now changed this to coincide better in Wii U and 3ds games hopefully.


Go to Page 0

Search this thread

Show all threads

Reply to this thread:

User Name Tags:

bold: [b]bold[/b]
italics: [i]italics[/i]
emphasis: [em]emphasis[/em]
underline: [u]underline[/u]
small: [small]small[/small]
Link: [url=http://www.google.com]Link[/url]

[img=https://www.hcs64.com/images/mm1.png]
Password
Subject
Message

HCS Forum Index
Halley's Comet Software
forum source