Choosing a compression method
Data compression reduces file size, limits storage use, and shortens transmission time. A method must first be chosen according to reconstruction requirements: must decompression preserve every bit, or is an approximation acceptable? This answer divides compression techniques into two categories: lossless and lossy compression. Both categories can use transforms, statistical models, and entropy coding, while their reconstruction requirements differ at the stage that produces the output.
The distinction affects both compression ratio and decompression output. Lossless compression guarantees a bit-for-bit copy. Lossy compression can produce a smaller file by permanently changing part of the data. The appropriate choice depends on the data type, required accuracy, and how the file will be used. For text and code, one changed character can affect meaning, while images and audio are also judged by perceptual quality.
: bit-exact reconstruction
Lossless compression represents data compactly while preserving every piece of information. Decompression restores exactly the same bit sequence, so the resulting file is equivalent to the source file. This includes byte order, control characters, and binary values when they belong to the input.
The core principle: using redundancy
Lossless compression detects and compactly codes . It can come from repeated sequences or unequal symbol frequencies. When a fragment can be reconstructed from the rest of the data, a shorter description can replace it. The decoder must know how to interpret that description so it can restore the original sequence. Lossless algorithms analyze two main types of redundancy:
- Repetitive Sequences: Data often contains repeating sequences of characters or bytes. Instead of storing the sequence every time it appears, a lossless algorithm can store it once and then use a short pointer or reference for subsequent occurrences.
- Statistical Predictability:Some symbols or values appear much more frequently than others. In English text, the letter 'e' is common, while 'z' and 'q' are rare. Lossless algorithms use this difference by assigning shorter binary codes to frequent symbols and longer codes to infrequent ones, reducing the average code length of the message.
Common lossless techniques
Several key methods form the basis of most lossless compression algorithms:
This is a simple compression technique. It replaces a long run of identical data with a single value and a count. In an image, a horizontal line of 200 white pixels can be stored as "200 white pixels" instead of listing every pixel. The method works well for computer-generated graphics, icons, and diagrams with large areas of solid color. The count has a defined range and representation, so short runs may produce no saving. For photographs, where long runs of identical pixels are rare, the gain is smaller.
Lempel-Ziv algorithms are used in many formats. DEFLATE combines LZ77 with Huffman coding and is used in ZIP, GZIP, and PNG. LZW, used in GIF and TIFF, builds a of recurring strings or byte patterns while processing data. When a known sequence appears, the algorithm emits a short reference to its dictionary entry. Depending on the variant, a reference can describe an index, a length, and a distance in the stream. In the sentence "The quick brown fox jumps over the lazy dog, the dog sleeps," the word "the" appears twice. A dictionary method can store the sequence once and use a reference for a later occurrence.
These methods analyze symbol frequencies and assign variable-length codes. Frequent symbols receive shorter binary codes, while rare symbols receive longer codes. is a classic example. If the letter 'E' appears most often, it might be assigned the code '0', while a rare letter like 'Z' might get '1110101'. Shorter codes for frequent symbols reduce the average length of the encoded message. Arithmetic coding represents a message as an interval in [0,1) and writes its position as a bit string. The average number of bits per symbol can then be fractional, while the complete encoded message still has an integral bit length.
Applications for lossless compression
Lossless compression is appropriate when data integrity is required. A changed bit can make a file unusable, alter its meaning, or affect the result of reading it. Checksums and error-detection mechanisms help detect damage, but they do not replace preservation of the data.
- Text and Documents: A legal document, a novel, or a simple email must be reconstructed exactly. The change of a single character could alter the meaning of a contract or a message.
- Computer Code and Executables: In a program, changing one bit in source code or an executable can change its behavior, cause an error, or affect its output.
- Databases and Spreadsheets: Financial records, scientific data, and any form of structured data require exact values. A small error could have serious financial or research consequences.
- Medical and Scientific Imagery: Some medical applications permit specialized lossy techniques. Archival of digital X-rays, MRI scans, and genomic data often uses lossless compression so that the data remains available for later analysis.
: approximate reconstruction
Lossy compression permanently changes part of the data to produce a smaller file. In multimedia, encoder parameters are chosen so that the difference remains acceptable for the intended viewer or listener and use case. The decoder reconstructs a signal from retained components, so the result depends on the codec, bitrate, and content.
The core principle: exploiting human perception
Lossy compression uses properties of human sight and hearing. With and models, encoders determine which components can be simplified at a selected quality level. These models are approximations that depend on the material and do not produce the same assessment for every listener or viewer.
- For Images: The human eye is more sensitive to changes in brightness (luminance) than to changes in color (chrominance). Algorithms such as JPEG can store color at a lower resolution than brightness through chroma subsampling. They can also reduce fine, high-frequency details, especially in complex parts of an image. Visible artifacts depend on edges, textures, contrast, and quantization level.
- For Audio: Lossy audio formats like MP3 and AAC use psychoacoustic models to identify masked components. A loud sound can make a quieter sound at a nearby frequency difficult to hear. The encoder can then reduce the description of that component; the result depends on the material and settings. At a low bitrate, simplifications can become audible as noise or distortion.
Common lossy techniques explained
Lossy compression is typically a multi-stage process involving transformation, quantization, and entropy coding.
Instead of working on pixels directly, algorithms like JPEG and MPEG first apply a mathematical transform, often the . In the mathematical model, the transform is reversible and reorganizes the data. A pixel block becomes a block of frequency coefficients describing low- and high-frequency components. In typical images, much of the energy is concentrated in a few low-frequency coefficients.
Quantization is the stage where data is permanently simplified. Frequency coefficients are divided by values from a quantization table, and the results are rounded to integers. In typical JPEG tables, higher frequencies use larger values, so some of those coefficients become zero. An encoder's "quality" parameter usually scales the quantization table. A higher value preserves more detail and produces a larger file, while a lower value simplifies more coefficients.
After quantization, the data contains many zeros and simplified coefficients. A lossless coder is then applied. Baseline JPEG uses Huffman codes, while the JPEG standard also defines arithmetic-coded processes.
Primary applications of lossy compression
Lossy compression is used for multimedia when limiting bitrate or file size matters more than preserving every bit. Image, audio, and video standards define their own models, transforms, quantization steps, and stream coding methods.
- Digital Photography: JPEG is a widely used standard for storing photos. It stores images at a selected quality in a fraction of the size of uncompressed data, which supports camera storage and photo sharing.
- Online video and streaming: Services such as YouTube, Netflix, and video conferencing use H.264 and H.265 video codecs to deliver streams at limited network bitrates. Encoder parameters can be adapted to resolution, frame rate, and available bitrate.
- Digital music: MP3 reduces the bitrate of audio, making it easier to store music libraries on portable devices and transfer files over the internet. Quality depends among other factors on bitrate and the recording material.
Generational loss should be considered. Saving a file again in a lossy format can discard more data and increase degradation. For repeated edits, it is useful to keep a source or lossless version and create a lossy export when needed. Each re-encoding operates on an already approximated result, so artifacts can accumulate.
Comparison of lossless and lossy compression
The table summarizes the main properties of both methods. The listed ratios are illustrative and depend on the data, codec, and settings. A ratio of 2:1 means that the encoded representation is half the reference size under the same size definition. Headers and metadata can change the measured ratio for short files. Dictionaries, model data, and bit padding can also contribute to the final representation.
| Feature | Lossless Compression | Lossy Compression |
|---|---|---|
| Data Integrity | Perfect, bit-for-bit reconstruction. Original = Decompressed. | Imperfect reconstruction. Original ≈ Decompressed. |
| Reversibility | Fully reversible process. | Irreversible process. Discarded data cannot be recovered. |
| Compression Ratio | Moderate (e.g., 2:1 to 3:1 for some data). Depends on data content. | High or higher (e.g., 10:1 to 100:1+). Adjustable through codec parameters. |
| Key Principle | Eliminates statistical redundancy. | Eliminates perceptually insignificant information. |
| Primary Use Cases | Text, programs, archives, databases, medical records. | Images, audio, video (multimedia). |