Previous Page | Next Page

by furrybob at 12:44 AM EDT on May 14, 2017

I spent some time in IDA this week researching the ADX encoder and format. Adxencd can create all the different ADX types, but I don't know if any of the rarer types can be found in the wild.

Does anybody happen to know of any? Maybe some ADX files that vgmstream wasn't able to play.

Also, I've reverse engineered a good part of adxencd and some of the newer ADX encoder. If anyone wants me to answer something related to my ADX research, e.g. maybe there's something that's unknown about the format, feel free to ask.

by bnnm at 5:09 AM EDT on May 14, 2017

@furrybob - Very nice, minor things I noticed:
- adcencd adds some beginning padding so loop start is block-aligned, recorded in the header.
ex. loop start=12 will add 20 padding, shifted to 32 (more info in adx.c). Is this padding played or skipped?
- FFmpeg's implementation contains some minor diffs:
EOF packet check (scale 0x8000), and some shifting during nibble decoding. Are these correct?
- There is a Japanese "ADX Decoder" around (1.30 here / also, similar to "HCA Decoder").
It has code to derive start/add/mult keys from keyworks/16b keys (like HCA), is it accurate?
Would be interesting to document or add for posterity.
Also guesses keys differently from guessadx.

@Knurek - I forgot to add .laac, derp. Next upload it is.
(since we have wav>lwav, ogg>logg I think aac>laac is the natural progression one would try for common exts; even though tri-Ace's has nothing to do with the actual .aac).

by marcusss at 11:27 AM EDT on May 14, 2017

Hi bnnm.

I ripped a game on XB360 called Blood Knights. For music it has a single FSB5 file ~96MB. I couldn't extract it with any program other than FsbExtractor.

Well that part is proably obvious since apparently inside aren't fsb but XMA files.. Problem is they have RIFF2 / WAVEfmt in the header but are not playable in VGMStream. Here are some samples

SAMPLE XMAs

Cheers

by bnnm at 11:39 AM EDT on May 14, 2017

Upload the original FSB or try fsb5_split. All other tools I've seen are faulty.

by marcusss at 12:25 PM EDT on May 14, 2017

Ok thanks. I will give it a try..

** It worked as they were in fact fsb5 files. Of course previously renaming them to fsb didn't work but glad this tool extracts them properly.

Thanks :-)

edited 12:29 PM EDT May 14, 2017

by furrybob at 1:36 PM EDT on May 14, 2017

adxencd doesn't actually contain an *.adx file decoder, so I couldn't tell you if the padding is skipped or not. It'll decode the encoded samples when encoding so it can use them as history samples, but that's mostly it.

I don't know what ffmpeg's doing with shifting the nibble there. I guess it's converting the unsigned nibble to a signed value somehow? The vgmstream decoder is accurate now, so you can use that as a reference.


Footer bytes:
This may be overly long, but here we go.
The 2 bytes at the beginning of each ADPCM frame contains two values. The first 3 bits specify which filter should be used for sample prediction. In standard type 3 encoding, this is always '0' because only one set of coefficients are used. In type 2 encoding, there are 4 predetermined filters that are hard-coded into the ADX format. Those first 3 bits specify which one to use. The remaining 13 bits contain the scale for that ADPCM frame. This is usually capped at 0x1000, giving the scale a range of 1 - 0x1000. The scale is decremented by one when it's stored in an ADX file, so the stored scale has a range of 0 - 0xFFF, leaving the 13th bit to be 0. However, the scale shouldn't be treated as a 12-bit value.

Adxencd has many hidden options that aren't displayed unless you send it the "-opt" argument. Even then, it has options that are completly undocumented. One of those is the "-G" flag. This flag will increase the max scale from 0x1000 to 0x1249(4681), or 0xFFFF / 14. With this scale, the entire range of 16-bit integers can be scaled to 4-bit signed nibbles. e.g. [-2340, 2340] will scale to '0', and each nibble could have originally represented any of 4681 different samples. Except for -7 and 7. Their range is cut in half and extends past the max 16-bit values. I don't know why they did it that way. It's also why you'll never see an '8' in the ADPCM data. It's just not used.

The ADX footer is marked by 0x8001 (Not 0x8000), followed by a 16-bit integer that specifies how many padding bytes there are past the footer. This should be a multiple of the frame size. Since the max possible filter is '3' (011b), a filter of '4' (100b or 0x80 extended to a byte) would be an invalid filter, and it would be invalid ADPCM data as well (I think). So it's used to mark the point where the audio data ends.

Back to the original question, technically FFmpeg's EOF check is not correct, although it works. This is assuming the 0x8001 marker doesn't change.


I haven't looked into ADX decryption yet, since it doesn't look like adxencd does it. However, I can help with verifying that the keys are correct. Due to the nature of ADX, an ADX file decoded and then reencoded will be extremely similar if the encoding and decoding implementations mirror each other and are correct. I've got an encoder here that matches adxencd's encoder. You could decrypt an ADX with a supected key, decode and reencode it, and compare the resulting ADPCM. If the key matches, the data should be very similar, but if the key is even slightly off, the reencoded audio should quickly diverge from the original audio. (I think)

Edit: I just tested verifying the xor keys. I used an ADX from Sonic and the Black Knight and decoded then reencoded the ADX using the key in vgmstream. The first few hundred bytes of the resulting ADX were 99% the same as the original ADX. Then I tried incrementing each element of the key individually. Each time the resulting ADX was vastly different from the original. So I guess that works, and the key for Sonic and the Black Knight is correct.

edited 2:22 PM EDT May 14, 2017

by AnonRunzes at 2:32 PM EDT on May 15, 2017

@furrybob - Now that you mention it, I have that ADX-like file(or rather, files stored inside one big .SM2 file) out of the Xbox version of Batman Vengeance. I`ll be digging one of its files up if you want to confirm my claims.

edited 3:39 PM EDT May 15, 2017

by Knurek at 2:37 PM EDT on May 16, 2017

Could anyone take a look at Steam versions of both New Tokyo Legacy games?

I'm guessing both games use OGG Vorbis, but the files are encrypted.
Given that the headers in both games are the same, it doesn't look like a very complicated encryption, probably some rolling xor type.

by Ultrafighter at 12:29 PM EDT on May 18, 2017

Hi Bnnm, I'm wondering if EALayer3 has already been implemented into VGMStream or not... Because if not it definitely wouldn't hurt to add such a codec to the plugin as numerous videogames use it, both really old ones and relatively new titles.
EA devs started using it more than 10 years ago (e.g. in Goldeneye: Rogue agent a.k.a. GoldenEye: Dark agent for GC or SSX: On tour from PS2, both sets are on Josh's servers BTW) and continued utilizing it in next-gen era (see Burnout: Paradise (PC/PS3) or Need for speed: Hot pursuit (PS3), these two rips can also be found in psf3 section of joshw.info). Older variants had "SCHl" codec identifier or better say container header. Such audio files IE with "SCHl" at start but using different codec (Electronic Arts 4-bit ADPCM (XA based) instead of non-standard MP3 (?)) were implemented years ago, those tracks can usually be played if you change given extension to *.asf, *.eam or *.sng. On the other hand newer EALayer3 files normally have *.snr or *.sns extensions.
Anyway here's decoder for EALayer3 files, it was made by author of DecUbiSnd. As expected it's a command-line based utility with a few options available, I once processed all aforementioned gamerips using just this tool and all resulting MP3s sounded fine.

I'd also like to post some other samples, maybe they'd be very easy to add to vgmstream. If not then it's no big deal to me after all. So here they're:

XAU files coming from Beat down: Fists of vengeance for XBox.
PS2 counterparts had the same extension and could be played in VGMstream right away. All *.xau tracks on XBox had to be extracted though (you can see typical wave file start at 0x40), I scanned all original XAUs with NovaSoft Extractor and played back all found WAVs as *.lwav ones, that's the way I ripped the game on that exact platform.
Probably implementation of XBox XAUs isn't a hard task for you and possibly the very same format is used in some other VG titles; if you don't see the point of such an effort I can understand that too.

PC/XBox RWS containers holding Microsoft 4-bit IMA ADPCM.
PS2 *.rws with Playstation 4-bit ADPCM were added quite some time ago but this format is also used on other platforms and codec is obviously not the same. For example Jacked (PC&XBox versions) along with Super-bikes: Riding challenge (PC port) use RWSs + IMA ADPCM and such files are currently unplayable. I managed to rip those troublesome RWS files with help of EkszBox-ABX v1.4 though (it extracts contained stream and adds RIFF header with codec number "11" or "69" to output).

If you want more diverse example files feel free to ask, for example Beat down uses XAUs for all its BGM (both clean tracks & VGM suites with ambient sounds on the background), ambience & even VO clips IIRC. I can also upload some Super-bikes samples really soon if those RWSs from Jacked are not enough for you to understand the audio format.
All the best!

by AnonRunzes at 4:29 PM EDT on May 18, 2017

Here is another .rws file, straight from the Xbox version of Burnout 2:
trk01.rar

edited 4:29 PM EDT May 18, 2017

Previous Page | Next Page
Go to Page 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271

Search this thread

Show all threads

Reply to this thread:

User Name		Tags: bold: [b]bold[/b] italics: [i]italics[/i] emphasis: [em]emphasis[/em] underline: [u]underline[/u] small: [small]small[/small] Link: [url=http://www.google.com]Link[/url] [img=https://www.hcs64.com/images/mm1.png]
Password
Subject
Message