Oddities between auCDtect and most vgmstreams by jonai at 6:31 AM EST on January 7, 2010
Hello,
do you know the tool auCDtect?? It is a command line tool to detect if a audio source was encoded lossy (ie. mp3 compression) or lossless.
Until now it was quite handy for me to verify my own CD-Audio rips and to find out if some official game-soundtrack cd's are encoded from lesser quality source material.
So I wanted to test it just for fun with my vgmstream ADX rips of 'The King of Fighters XI'. I expected that the ADX files would have a high percentage of probability that they are lossy encoded, because from what I understand ADX is a lossy format. So imagine my amazement when I found out that AuCDtect says in its logs that almost all of the tested tracks should be 100% or 99% probably correct lossless CDA tracks. So now I wanted to know more and I tested some crappy (in terms of sound quality not in terms of quality of the content...) tracks from the 'Sin and Punishment 2' tracks, with the same result. Just to verify this: auCDtect can verify lossy tracks, If I encode anything to mp3 and test the tracs.
What I wanted to know is something about the quality of the inner workings of ADX and ADPCM, because this would mean that they work in another way then mp3 and/or are of higher quality than other lossy formats (or that the detection is broken for formats like ADX...). Also I would want to know If you could produce similar results with your vgm-stream rips.
Maybe you can bring me some enlightenment of the inner workings of ADX and ADPCM, because Wikipedia was not very helpful in this topic.
The detection is most likely specific to MP3, maybe looking for a sharp (though varying) frequency cutoff. ADX has a delta value per sample so there is no inherent limit in frequency that would create such a cutoff. I don't know much about MP3, there might be other suspicious things that are easy to check for.
As I understand it ADX has almost nothing in common with MP3, it is very simple and strictly time-domain based. ADPCM is based around the idea that you can predict what the next sample will be based on what the previous samples are (in ADX the previous two samples are weighted and summed), then all you should need to store is the error (aka delta) between the predicted sample and the actual sample. ADX (and most ADPCMs in use in games) uses 4 bits for this error, representing -8 to 7. The error is scaled by a 16-bit scale value that's provided every 32 samples. This means that it can only represent 16 possible sizes of error exactly (-8*scale, -7*scale, ... 7*scale) for those 32 samples, which is where the loss comes in. I haven't looked at encoding methods for ADPCM, but I assume it works to minimize the total amplitude it throws out in a particular 32 sample frame. Thus certain fine (low amplitude) details are lost.
MP3, on the other hand, does (I believe) the Discrete Cosine Transform, which transforms a frame (500-something samples) into a set of cosines of varying amplitude and frequency which would combine to produce that frame. It then throws out certain cosines which are hard to hear, largely this is at the expense of high frequencies but it also removes low amplitude frequencies near higher amplitude ones (as the high amplitude apparently masks it). Lots of other psychoacoustic stuff and I'm sure I'm misrepresenting it. The frequencies to be kept are then stored, I believe with a Huffman-y entropy coding, and since much information was discarded there's much less to store.
While I can imagine something that would test for ADPCM, I don't really see much point in pursuing it. "Lossy" really only means something relative to a particular source material, and when the source material is a game the digital rip truly is lossless.
Oddities between auCDtect and most vgmstreams by jonai at 9:55 AM EST on January 7, 2010
Thank you for your reply,
off course are relative to the source (the game itself) direct game rips lossless. As I mentioned I want to try some tests out of curiosity to learn how game music and game music soundtrack cd's are made. Especially those 'remastered' versions (Rockman ZX remastered tunes comes to mind) and all those 'Original Sound Version' CDs. I thought previously that could be a method to detect low quality bootlegs or low quality CDs in general.
From what I understand from your reply regarding ADPCM, that would mean that it is better suited for storing a continuous signal or a continuous changing signal (that should be the most old game soundtracks), while mp3 would cut down the exact same signals. Maybe you already know the term 'equal to CD quality' when it comes to the discussion about quality of the method of storing audio signals (not the quality of the contents of course, if there is not noise that is obviously generated because of the compression method). I think that I can now assume that ADPCM is at least 'equal' to CD-Audio quality if not better than mp3 for the special purpose of storing signals like game-music.
sorry for the long post. OFF TOPIC PS.: I have to admit that this game music ripping is a very fun thing for me to do, because I can learn so much in the process of doing so. fare well until then.
I think that I can now assume that ADPCM is at least 'equal' to CD-Audio quality if not better than mp3 for the special purpose of storing signals like game-music.
Kind of like comparing apples and oranges? Thanks, hcs. Interesting stuff. Jonai, you may want to try Informer:
From what I understand from your reply regarding ADPCM, that would mean that it is better suited for storing a continuous signal or a continuous changing signal (that should be the most old game soundtracks), while mp3 would cut down the exact same signals.
I'm confused by this. ADPCM and MP3 cut down different things, I don't think anything I said suggested one was better than the other for anything. I don't know what you mean by "a continuous signal" or "a continuous changing signal", that seems to describe any sound.
ADPCM is used because it's a good compromise between size and decode complexity, while MP3 takes somewhat more computation to decode though it can achieve much smaller sizes for the same perceived quality. The tradeoff is acceptable given the size of game discs (and even in many modern cartridges) and the fairly weak decoder hardware.
Oddities between auCDtect and most vgmstreams by jonai at 7:07 AM EST on January 8, 2010
@SmartOne Thank you very much, this information was very helpful for me. I will do some more tests with this one.
@hcs Sorry if I confuse you with my wording, English is not my best language. I will try to illustrate what I mean when I have done some more tests. The only purpose of this is just the exploration of game-music (on the technical side) and my entertainment.
to illustrate what I said in the first post here is the log of auCDtect from 'The King of Fighters XI' This are the files that are not 100% or 99% CDDAs, note the two tracks that look like mpeg.
auCDtect: CD records authenticity detector, version 0.8.2 Copyright (c) 2004 Oleg Berngardt. All rights reserved. Copyright (c) 2004 Alexander Djourik. All rights reserved.
Processing file: [Arcade - 07 - Queen (SHA-V - SNK Playmore).adx.wav] This track looks like CDDA with probability 54%
Processing file: [Arcade - 17 - Continue (SHA-V - SNK Playmore).adx.wav] This track looks like MPEG with probability 97%
Processing file: [Arcade - 27 - Awakening (SHA-V - SNK Playmore).adx.wav] This track looks like CDDA with probability 54%
Processing file: [Arcade - 28 - Ending - Rain (SHA-V - SNK Playmore).adx.wav] Could not qualify the source of this track.
Processing file: [Arcade - 30 - Ending - Rest (SHA-V - SNK Playmore).adx.wav] This track looks like CDDA with probability 92%
Processing file: [Arcade - 31 - Ending - Conference (SHA-V - SNK Playmore).adx.wav] This track looks like CDDA with probability 58%
Processing file: [Arcade - 35 - Ending - Collection (SHA-V - SNK Playmore).adx.wav] Could not qualify the source of this track.
Processing file: [Arcade - 36 - Ending - Parting (SHA-V - SNK Playmore).adx.wav] This track looks like CDDA with probability 42%
Processing file: [Arrange - 11 - Pure World (Koji Takata, Masuo Okumra & Masanori Kuki).adx.wav] This track looks like MPEG with probability 100%
Processing file: [Arrange - 18 - Service (Koji Takata, Masuo Okumra & Masanori Kuki).adx.wav] This track looks like CDDA with probability 42%
Processing file: [Arrange - 23 - Groaning World (Koji Takata, Masuo Okumra & Masanori Kuki).adx.wav] This track looks like CDDA with probability 92%
Processing file: [Arrange - 28 - Ending - Rain (Koji Takata, Masuo Okumra & Masanori Kuki).adx.wav] This track looks like CDDA with probability 58%
Processing file: [Arrange - 31 - Ending - Conference (Koji Takata, Masuo Okumra & Masanori Kuki).adx.wav] This track looks like CDDA with probability 49%
Processing file: [Arrange - 35 - Ending - Collection (Koji Takata, Masuo Okumra & Masanori Kuki).adx.wav] This track looks like CDDA with probability 85%
Processing file: [Arrange - 37 - See You (Koji Takata, Masuo Okumra & Masanori Kuki).adx.wav] This track looks like CDDA with probability 54%
Drug here by a Google search for "aucdtect adpcm", it seems like this is the only useful result. I too was curious and decided to test some adx files from some games, and they turned up as being analyzed as "CDDA with probability 100%" I just wanted to leave the information, in case someone else in the future runs across this, that auCDtect will correctly identify not just MP3, but also newer codecs, including Opus, AAC, and OGG Vorbis at their highest bitrates as being lossy. I guess that it's looking for patterns commonly used by most codecs using a psychoacoustic model.