eC!

Social top

English

A Survey of Audio Coders for Electronic-Art Music

Coded digital audio is undeniably a necessity in the iPod age. Not long after making the switch from records to compact discs in our personal music libraries, we have grown accustomed to a lesser fidelity via audio coding in favour of portability, accessibility, storage-space and web/pod-casting capability among other advantages. In the last decade or so, millions of music lovers worldwide have transformed their CD, record, and cassette collections into digital data on their computer hard drives (often not with the best choice of coding); various low-datarate coders, such as Real audio have allowed audio webcasts as early as the 14.4 kbps age; and satellite radios, such as XM and Sirius, allow us to tune-in to low-bitrate audio streams across the continent.

For the electronic musician, audio coding plays additional important roles: (1) often compressed-audio files are used as sound source, either field-recordings done on a minidisc or other devices, sounds downloaded from the internet, or recorded sounds shared with other musicians via email, p2p or bit torrents. (2) Some musicians use the web for live collaboration via internet telephony and messaging services (the likes of Skype and Windows Messenger). Musicians also use coded sound to (3) post their music online, (4) podcast it, (5) send it to competitions or promote it online in any other way.

This survey examines several popular audio codecs and compares their accuracy and resulting sound quality. There are numerous audio compression formats, some lossy and some lossless, each approaching the task of reducing the digital-audio file’s size differently. Lossless compression formats such as Apple Lossless Audio Codec (ALAC) or Free Lossless Audio Codec (FLAC), among others, stay true to the original PCM source data and are assessable primarily by their encoder/decoder efficiency and performance, while lossy formats should, additionally, be evaluated by fidelity and accuracy, which are dependent mainly on the format’s psychoacoustic model, time-to-frequency mapping, and bit allocation.

Most psychoacoustic models of lossy compression formats are optimized for tonal (of tones) sound material, with little, if any, instantaneous radical spectral or dynamic changes, and occupy the spectrum most prominently between 0 and 5 kHz — the rest of the spectrum being taken mostly by related partials, not independent content. The psychoacoustic models of these compression formats rely on time-to-frequency mapping, which reduces the temporal resolution in favor of higher frequency resolution, therefore causing inefficiency in coding short-attacks and transient sounds. These characteristics make lossy audio coding far from ideal for electronic art-music and sound art.

This survey explores and evaluates some of the losses in several popular lossy codecs including mp3, m4a (AAC), mpc (Musepack), oma (Sony ATRAC3plus), ogg (Vorbis), and wma (Microsoft). All sounds will be compared in a low bitrate, 128 kbps (stereo, CBR or ABR), and a high one, 256 kbps. The choice of these two is not accidental; 128 kbps (also known as near-CD quality) is presently the bitrate of files purchased on iTunes in the m4p format (an m4a with DRM protection), and is also quite common in audio streaming. Additionally, this low datarate exposes the deficiencies of the psychoacoustic models quite effectively; to accentuate these deficiencies, the sound content in this survey is designed specifically to attack the vulnerabilities of the codecs. The higher bitrate, 256 kbps, is arguably a sufficiently high datarate for audio coding while still saving a significant amount of storage space. The usefulness of datarates 320 kbps and higher is questionable because lossless compression formats average bitrates that are not much higher (about 450 kbps) without any data loss (and therefore no need for a psychoacoustic model). Additionally, 256 kbps is the default bitrate used in Sony Hi-MD minidisc field recorders in the ATRAC3Plus format, which is one of the formats examined in this survey.

Since lossy compression relies on the transformation from the time domain to the frequency domain, compromising one resolution for the other, this survey examines each domain individually.

Test 1 — Time Domain

The first test examines the transient responses of the different coders. The coders are given the simple task of coding a 0.5 Hz square wave, which switches the amplitude’s polarity twice every cycle (therefore sounding a pulse every second). The switch is immediate (one sample) in the original PCM format and the coders are examined for (1) whether they retain the sharpness of the switch and (2) whether they distort the waveform and add noise.

The images below are in a single-sample resolution and are not always representative of fidelity; they demonstrate only the polarity switch but not necessarily the pulse in its totality, the overall sound or added noise as you shall find when listening to these examples. Click the images to listen and read the comments on the right-hand column. Despite the connecting lines between the individual samples in these images, in reality there is no such graduation between the amplitude values of adjacent samples; nevertheless, some sloping will occur in the conversion to acoustic energy at the loudspeaker.

To avoid player and format compatibility problems, all audio examples were reconverted to wav format and therefore should be playable on any system.

All coders used here are the latest available versions as of May 2007.

It is preferable to listen to these examples on a pro or semi-pro system with a wide, flat frequency response. If you are using headphones, please watch the levels!

In some cases it may take a few listen-attempts to segregate the noise; however, once heard, it is almost impossible to “unhear” it.

Codec Image Notes
wav file (PCM), 44.1 kHz, 16 bit (Uncompressed) 1411kbps
image
This is the uncompressed original signal; listen to the immediacy and dryness of the pulses; compare with the other examples.
flac (Lossless) 9 kbps
image
This is an exact replica of the original sound. And yes, 9 kbps is correct; a very simple signal such as this one does not require much storage space to be accurately described in a lossless coder.
Codec 128 kbps 256 kbps Notes
mp3, LAME, CBR
image

image

Although to a lesser extent in 256 kbps, in both cases the pre-echo is very noticeable (instead of a “tk” onset you can hear “ft”); it is also very inconsistent, sometimes being longer than others.
Additionally, listen to the oscillated tones, most prominent 1378 Hz (around F6), which actually come out stronger in 256 kbps.

mp3, LAME, ABR image image The average-bit-rate coding here does not demonstrate an advantage over constant-bit-rate; the pre-echo and added tones are just as noticeable.
mp3, Blade, CBR image image

The transient response of this encoder is even worse than that of LAME; however, it does not introduce any oscillated tones at 128 kbps
the 256 kbps version introduces a very weak (nearly inaudible), extremely high-frequency tone.

m4a, ABR image image This format does quite a good job. The pre and post-echos are quite insignificant in both the 128 and 256 kbps versions.
The added noise is spread more widely and granulated, which makes it less noticeable than the noise in mp3 coding. Listen carefully and you’ll hear, in both cases, some very weak tones and a general weak rattle in the background.
ogg VORBIS, ABR image image Despite looking nice in the amplitude time-line, this format does not deal well with the LF square wave. In both bitrates the pre-echo is long, loud, and the post echo has pitch.
Also, there is a very strong 43 Hz oscillated tone (around F1) among a few others.
oma (SONY), ATRAC3plus image image Unfortunately, here too the promising image is misleading. The pre-echo in both bitrates is long and loud (hear the “woosh” sound before each pulse).
However, this format is by far the cleanest of sustained noise.
wma (Microsoft), WMA 10 Pro, CBR image image This format does an excellent job! The pre-echo is tolerable and there is virtually no added sustained noise.
The main weakness here — although only serious in the low bitrate version — is a slight post-echo, heard as a slight reverberation which changes the spatial quality of the pulse (more depth than the original signal).
mpc (Musepack), VBR 112-152 kbps
image
232-268 kbps
image
This format is quite good as well in terms of pre and post echoes but it also adds some oscillated tones — most prominently at 1335 Hz (around E6), although not as strongly as mp3 or ogg.


Additional points to consider:

  1. The wav format itself (PCM) is not error-free; the very concept of sampling introduces distortion, primarily by amplitude quantization (round-off noise), clipping, LSB oscillation, and aliasing; however, as this format is exclusively time-domain based and does not utilize a psychoacoustic masking model, its errors are much more predictable and easily handled in the codec than the other coding methods. When possible, it is recommended to use a 24-bit resolution or higher to reduce the round-off and LSB noises (or to send them outside the audible dynamic and frequency ranges). Presently, among the lossy compression formats surveyed, only the Microsoft WMA 10 Pro supports 24 bit coding; lossless compression supports it as well.
  2. When the audio signal is static in nature, simple, or with long silences, lossless coding may be advantageous even in terms of storage space. The lossless flac file in this example is almost X30 smaller than the (very) lossy 256 kbps mp3 format.
  3. It appears that the best performer in this section of the survey is Microsoft’s 256 kbps WMA 10 Pro format, with its 128 kbps counterpart at second place. The 128 kbps m4a format (the iTunes format) is quite adequate for the low bitrate as well; its 256 kbps version, however, does not offer a significant improvement in this example. The performance of all the other codecs is less than adequate, with ogg being the worst in this instance. These results may not represent other signal types.
  4. There are many other issues to consider when choosing a codec, among them are player compatibility and availability. Mp3, for instance, has been, by far, the most widely used, while clearly not the best choice for fidelity. This survey focuses on reproductive accuracy alone.

Test 2 — Frequency Domain

In the second test the coders are given the task to code a white noise signal, which was passed through a digital HPF with a 10 kHz cutoff frequency and a moderate slope; this allows us to examine how the coders cope with a spectrally irregular signal in their weaker areas.

Image

wav file (PCM), 44.1 kHz, 16 bit

This is the original uncompressed file.

(Uncompressed) 1411kbps
image

flac
(Lossless) 1165 kbps
The image and sound are identical.

This is an exact replica of the original sound, and therefore so is the spectrograph. The highly random content requires a high bitrate and makes the compression ratio almost ineffective (83%).

mp3, Fraunhofer,

encoder-speed / quality

The Fraunhofer coder allows fast encoding and high quality encoding; while the fast option may be useful for streaming, you must use the high quality encoding when generating a permanent file. However, both versions strongly distort the original signal. In addition to cutting off all frequencies above 15.5 kHz, there are very visible holes in the spectral image. Listen to the “metallic” aliasing noise and to the timbral irregularities in the highest frequencies. 

Note
: as higher frequencies tend to sound more spatially centered, you may perceptually scan the spectrum by narrowing your spatial focus (use headphones).

128 kbps fast encoding
image
128 kbps high quality encoding
image
128 kbps 256 kbps

mp3, LAME, CBR

The LAME coder does a better job than the Fraunhofer coder. The distortion is much less significant, although still audible in both the 128 and the 256 kbps versions. Listen carefully to timbral variations in the higher areas of the spectrum. The overall timbre of the lower bitrate version also sounds significantly less bright than the original signal.

image image

mp3, LAME, ABR

The average bitrate method does not provide better results here as its usefulness relies on varying the bitrate according to the complexity of the content. In this case, the sound is uniformly complex throughout the example, therefore the bitrate would not vary at all.

mp3, Blade, CBR

In 128 kbps, the signal sounds surprisingly adequate; there are some audible instabilities in the high frequencies and an overall drop in “brightness” due to the coder's cutting off frequencies above 14 kHz (very low!), but overall the loss of high frequencies is worth the decrease in audible artifacts in comparison to LAME.

The 256 kbps version is very similar to that of LAME but with a little more audible artifacts.

image image

m4a, ABR

This coder does an excellent job - both in the low and high bitrates. In both cases the distortion is practically inaudible, and the overall timbre sounds identical to the source. A good ear may pick up very subtle timbral variations in the highest area of the spectrum. The main advantage of the higher bitrate is its full-spectrum coverage; however, cutting-off frequencies above 17.3 kHz (in 128 kbps) is not a problem for most users.

image image

ogg VORBIS, ABR

The 128 kbps ogg file covers a wider area of the spectrum than its m4a counterpart (almost as high as 19 kHz) but with less accuracy. The spectrograph displays a less gradual decrease in intensity towards the lower frequencies, therefore a distortion of the HPF's original slope; as seen in the spectrographic image, the division into frequency bands is quite coarse. A low level aliasing distortion makes the overall sound a bit more “metallic” and less pleasant.

The 256 kbps is extremely close to the original sound, but a careful ear will notice a subtle timbral difference, the original signal sounding a little “fuller”.

image image

oma (SONY), ATRAC3plus

The 128 kbps SONY format cuts off at 15.3 kHz, which, like the Blade mp3 format, is perhaps too low for professional uses; the 256 kbps (the minidisc recording format) covers the entire audible spectrum. Overall this format deals well with this complex signal. In both cases aliasing and other forms of distortion are not a problem. The lower bitrate is less "bright" due to the loss of frequencies above 15.3 kHz; the 256 kbps version is nearly perfect.

image image

wma (Microsoft), WMA 10 Pro, CBR

Like the SONY format, this Microsoft coder cuts off at 15.3 kHz in the 128 kbps setting and covers the full spectrum in 256 kbps.

Unlike in ATRAC3Plus, aliasing is a significant problem in both bitrate settings, as is a spatial distortion caused by reverberation. While this format performed superbly in the time domain, it adds very audible artifacts here.

image image

mpc (Musepack), VBR

Although to a lesser extent in the higher bitrate, both settings introduce some timbral instabilities in the high frequency range and a very high-pitched metallic ring (unpleasant).

112-152 kbps
image
232-268 kbps
image

Additional points to consider:

Final Conclusions and Recommendations

  1. Make lossless coding your first choice! If your sound is spectrally simple, static in nature, or has long periods of silence, lossless coding may even save you storage space when compared with lossy formats. On the other hand, if your sound is spectrally complex, includes many transients, or both, the loss in fidelity may be too great a price to pay for the space saving offered by lossy coding.
  2. In instances where you must compress your sound, m4a (AAC) would be your best bet among the coders surveyed here — both in its low and high bitrates; its performance was extremely effective in both tests. Additionally, being the iTunes format, m4a is widely used, strongly supported and frequently updated.
  3. Mp3 is not among the strong performers of this survey, but its history, global player-compatibility, and accessibility give it dominance. In many instances you will be asked to provide your works in mp3 format; you must choose your coder and settings very carefully. Among the surveyed encoders, LAME produced the best (least bad) results, with one exception: the low bitrate encoding of a spectrally complex signal which was, perhaps, a bit better with Blade. If you are forced to use 128 kbps, and especially if your digital sound is not rich in transients, you may be better off with Blade instead of LAME. The only way to be certain, naturally, is to try them both.
  4. This survey did not examine the effectiveness of using a VBR coding. As a general rule, allowing an encoder the flexibility of varying the bitrate may prove useful on sounds with large dynamic and spectral ranges (such as classical music, experimental music, electronic art-music, sound art, etc.); however, this additional decision-making process may also introduce new problems. Most contemporary codecs, nonetheless, allow effective usage of both CBR and VBR; some codecs use VBR or ABR (a variable bitrate encoding with a preset average rate) by default; among them ogg Vorbis, m4a, and mpc.
  5. When recording with a Hi-MD minidisc, use PCM recording when possible, especially if you are recording a signal with many transients; however, if you must record continuously for a period longer than 1.5 hours (the time-limit on a 1GB disc) you may use ATRAC3Plus at 256 kbps (Hi-SP), which is quite effective throughout the entire audible spectrum.
  6. A quick look on the tests’ results shows no apparent advantages to using the Sony, Musepack and Vorbis formats over m4a. The Microsoft format demonstrated a slight advantage in the time domain test over the m4a but with a bigger penalty in the frequency domain. Nonetheless, if your signal is mostly made out of transients with little sustained or reverberated sounds you may find the WMA 10 Pro format advantageous.
  7. These two tests attack the codecs’ vulnerabilities in order to demonstrate how the codecs deal with the highly-irregular content of electroacoustic music as generally and as objectively as possible. However, each sound work may introduce its own “issues” and your best option would be to try-out several coders and evaluate their performances in the time and frequency domains.

Recommended Bibliography

Bosi, Marina and Richard E. Goldberg. Introduction to Digital Audio Coding and Standards. Boston, MA: Kluwer Academic Publishers, 2002.

Guzmán, Luis E. G. “Compresión Perceptual de Audio Digital.” Tesis de Maestro en Ciencias en Ingeniería Elélctrica, Universidad Michoacana de San Nicolás de Hidalgo, November 2005.

Hauge, Knut O. and Svein B. Skogly. “Analysis of Audio Coding Algorithms for Networked Embedded Systems.” Masters Thesis in Information and Communication Technology, Agder University College & The University of Queensland, May 2004.

Tsutsui, Kyoya, Hiroshi Suzuki, Osamu Shimoyoshi, Mito Sonohara, Kenzo Akagiri, and Robert M. Heddle. “ATRAC: Adaptive Transform Acoustic Coding for MiniDisc.” October 1992.

Various. “Discussion of Audio Compression.”
A collection of documents posted by individuals, including technical data, tests, reviews, recommendations, and general chatter.

Various. “MP3 Tech.” 1996.
A site dedicated to the mp3 standard, including information about upcoming audio compression techniques, tests, source codes, and more.

With thanks to Kevin Austin and Tim Ramsay for their useful suggestions.

20 May 2007

Social bottom