Because XMA was based on the lossy WMA format. Microsoft shilled this lame format(not XMA) as something that would surpass most audio formats(even MP3), and the result was nothing but disaster out of boasting the format(resulting in many WMA variants being developed, including the format itself) - and thus, XMA was born when Microsoft thought it was a good idea to make a WMA variant on their game console. Therefore it's lossy and compressed. You can also decode XMA files with ffmpeg as well.
For instance, the Windows version of Terraria uses 48kbps WMAv2 audio for its music.
The only way to get decent music from the Windows version is to steal the music package from the Mac version, which uses MS ADPCM for the audio, which is considerably higher quality.
The Linux version uses high quality audio as well, but the audio pack isn't compatible with XNA. Possibly because they may be using Ogg Vorbis there?