Audio Compression

Digital Sound: Representation and Data Size

Music streamed through Spotify, podcasts, and film soundtracks, including Netflix content, are stored as digital data. Their size must be limited so that a library fits on a portable device and high-quality audio can be sent over a cellular network. Audio compression is one way to do this.

Uncompressed audio requires substantial storage because an analog signal has to be represented by samples. provides this representation. PCM stores samples rather than reducing their number, so size grows with sampling frequency, bit depth, channel count, and recording time. It has two stages:

Sampling: The analog waveform is measured at regular intervals. CD-quality audio uses 44,100 samples per second. This rate follows the Nyquist-Shannon sampling theorem and supports representation of frequencies up to about 20 kHz.
Quantization: Each sample amplitude receives a numerical value. CD-quality audio uses 16 bits per sample, giving $2^{16}$ , or 65,536 levels. This number of bits is the bit depth. A sample is rounded to one of those levels, so PCM itself can contain quantization error.

The size of a standard three-minute uncompressed stereo recording can be estimated as follows:

$44,100 \text{ samples/second} \times 16 \text{ bits/sample} \times 2 \text{ channels (stereo)} \times 180 \text{ seconds}$

$\approx 254,016,000 \text{ bits}$

$\approx 31.75 \text{ megabytes (MB)}$

More than 30 megabytes per recording quickly increases the storage required by a library of several albums. The corresponding bitrate can also exceed the capacity of some internet connections. The calculation distinguishes bits from bytes and decimal units from binary units. Lossless formats such as FLAC often reduce size by about 50 percent without discarding samples, depending on the material. can reduce file size by 90 percent or more at selected settings.

Perceptual Audio Coding: Modelling Audibility

Lossless compression removes redundancy while preserving every sample. Lossy coding uses properties of human hearing to limit information with lower perceptual importance. This is the subject of . Codecs such as MP3 and AAC use psychoacoustic models for perceptual audio coding.

Codecs such as MP3 and AAC use the hearing threshold and auditory masking to control quantization noise. An encoder model approximates hearing measurements, so the masking threshold depends on level, frequency, duration, and the selected model.

Principle 1: Absolute Threshold of Hearing (ATH)

Hearing sensitivity depends on frequency. The describes this level across the spectrum. Sensitivity is usually greatest from 2 to 5 kHz, while the threshold is higher at very low and very high frequencies. The ATH curve is a reference; the encoder also accounts for maskers in the current segment.

An encoder estimates the threshold for each analyzed segment. A component below that estimate may be omitted because it is undetectable under the relevant listening conditions. The amount of this reduction depends on the signal and encoder settings.

Principle 2: Auditory Masking

Auditory masking is a reduction in the audibility of one sound caused by another, usually louder, component. A estimates the effect so that quantization noise can remain below the masking threshold. Masking has frequency and temporal forms:

Frequency Masking (Simultaneous Masking)

A loud component can mask a quieter sound that occurs at the same time and in a nearby band. Trying to hear a whisper during a loud rock concert is a clear example. A masking curve is strongest near the masker frequency and usually falls as the frequency distance increases.

Temporal Masking (Non-simultaneous Masking)

Masking also has a temporal dimension. A loud sound can reduce the audibility of a quieter component immediately before or after it.

Post-Masking: After a loud, abrupt sound such as a cymbal crash, hearing sensitivity recovers gradually. Quieter sounds immediately after it can be masked for up to 200 milliseconds.
Pre-Masking: A loud sound can reduce the perception of a quieter component occurring a few milliseconds before it. Many descriptions use about 5 milliseconds. A 20-millisecond value can describe an encoder analysis window and is not a general limit for pre-masking.

An encoder does not treat these effects as unconditional permission to remove data. It keeps a margin because listening conditions and individual hearing change audibility. For short transients, it can switch to a shorter analysis window to limit pre-echo, an audible coding artifact that appears before an impulse.

MP3 Encoding: Main Stages

MP3 combines psychoacoustic analysis with a hybrid filter bank, quantization, and entropy coding. Implementations can arrange internal operations differently, while the roles of the stages remain similar.

Framing: The input PCM stream is divided into frames. The encoder also uses overlapping analysis windows to preserve continuity at frame boundaries and respond to signal changes. In MPEG-1 Layer III, one frame represents 1152 audio samples, and some data can use a bit reservoir shared with earlier frames.
Hybrid Frequency Transformation: MP3 uses a hybrid filter bank: a polyphase analysis filter bank is combined with a switchable . This representation describes energy in bands where masking is modelled. The filter bank contains 32 polyphase subbands, while long and short MDCT windows provide different time and frequency resolution for steady signals and transients.
Psychoacoustic Analysis: In parallel, the signal is processed by a psychoacoustic model. The model identifies masking components and estimates a masking threshold for each frequency band in the current segment. Its output is a map of allowable quantization noise that guides bit allocation. The model can distinguish tonal and noise-like components, and it runs in the encoder without adding the same analysis burden to the decoder.
Quantization and Bit Allocation: This is the lossy stage. Frequency coefficients are quantized with a precision selected from the bit budget and the psychoacoustic model:
- Components for which more quantization noise is allowed can be represented with lower precision, and some coefficients can become zero.
- Components that matter more to perception receive more bits. The encoder chooses parameters so that quantization noise stays close to or below the masking threshold in each band.
Entropy Coding (Huffman): The quantized coefficient stream is encoded losslessly with Huffman code tables. More frequent values receive shorter codes, reducing the number of bits before the MP3 frames are written. This stage does not discard further information; it stores the already quantized values in a shorter representation. MP3 can also use a bit reservoir across frames, and each frame carries side information needed by the decoder.

From the decoder's perspective, the listed stages are reversible except for the earlier lossy quantization. The decoder reads each frame, decodes the Huffman codewords, applies the stored scale factors and frequency reconstruction, and then synthesizes PCM samples. It therefore does not need to rerun the encoder's psychoacoustic model. This division keeps decoder work predictable even though encoder analysis can be comparatively expensive. The stream also carries side information that tells the decoder how to interpret each coded region, including scale factors, block choices, and the locations of coded spectral values. The result is approximate PCM whose audible differences depend on the selected bitrate and encoder settings in practice.