It looks like file is just so loud in the parts that the guesser checks that the wrong key works better!
Loading a key from an external file is a better idea anyway, thanks for implementing it. I was caught up in hubris, I wanted vgmstream to be able to do everything by itself.