Evaluating Compression: The Three Pillars of Performance

Once we understand that data can be compressed, the immediate question becomes: how well did we compress it? The world of compression is filled with hundreds of different algorithms, each with its own strengths and weaknesses. Choosing the "best" one is not a simple task, as it almost always involves a trade-off. To make an informed decision, we must evaluate algorithms based on a set of objective and subjective criteria.

The performance of any compression method can be judged by three primary factors: its efficiency in reducing size, the speed at which it operates, and the quality of the final, decompressed result. While some criteria are easy to measure with simple mathematics, others, especially quality, can be far more elusive and dependent on the human observer. This section will explore the key metrics used to compare compression methods and understand the critical trade-offs involved, particularly the relationship between file size and perceptual quality.

Metric 1: Measuring Size Reduction

The most direct and fundamental goal of compression is to make data smaller. The effectiveness of this reduction is measured in several related ways, each offering a different perspective on the algorithm's performance.

Compression Ratio

The most common metric for expressing compression efficiency is the compression ratio. It directly answers the question: "How many times smaller is the compressed file compared to the original?"

For example, imagine you have a 100 megabyte (MB) raw image file. After compressing it with a lossless algorithm like PNG, the resulting file is 40 MB. The compression ratio would be:

This is typically expressed as a ratio, 2.5:1, and read as "two-and-a-half to one". It means the original data was 2.5 times larger than its compressed form. If you were to compress the same 100 MB image using a lossy algorithm like JPEG, you might achieve a compressed size of only 10 MB. The compression ratio would then be:

Or 10:1. A higher compression ratio always signifies a greater reduction in file size and thus a more efficient compression in terms of space saved.

Percentage Reduction

Another intuitive way to express efficiency is by the percentage of space saved. This answers the question: "By what percentage was the original file reduced?"

Using our JPEG example (100 MB down to 10 MB):

A 10:1 compression ratio is equivalent to a 90% size reduction. Both metrics describe the same outcome but from a slightly different angle.

Average Bit Rate (Bits per Symbol)

This metric connects the practical results of compression back to the theoretical limits defined by Information Theory. Instead of looking at the entire file, it measures the average number of bits used to represent a single fundamental unit, or symbol, of the original data. A "symbol" could be a character of text, a pixel in an image, or a sample of an audio waveform.

For example, a standard 24-bit color image uses 24 bits for every pixel. If, after lossless PNG compression, the file size implies that, on average, only 14.5 bits are being used per pixel, then the average bit rate is 14.5 bits/pixel. This metric is extremely useful because it can be directly compared to the entropy of the data source to judge how close an algorithm is to the theoretical best-case compression.

Metric 2: Speed and Resource Consumption

A highly effective compression algorithm is of little practical use if it takes days to compress a single file. The speed of operation is a critical factor, especially for real-time applications. We typically consider two types of speed:

• Compression Speed: How quickly an algorithm can take an uncompressed file and produce a compressed output. This is crucial for applications like live video broadcasting, where data must be compressed on the fly before being sent over the network, or for archiving large volumes of data where time is a factor.
• Decompression Speed: How quickly an algorithm can take a compressed file and reconstruct it. In many scenarios, this is even more important than compression speed. A file might be compressed only once, but it will be decompressed every time a user wants to view, play, or use it. Fast decompression is essential for a smooth user experience, whether it's quickly loading images on a webpage or playing back a high-definition video without stuttering.

There is almost always a trade-off between speed and compression ratio. More complex algorithms like bzip2 or LZMA (used in 7z archives) can often achieve better compression ratios than simpler ones like gzip's Deflate, but they are significantly slower to both compress and decompress. Furthermore, these processes consume computational resources, primarily CPU time and system memory (RAM). An algorithm that requires a large amount of RAM may not be suitable for low-power devices like smartphones or embedded systems.

Metric 3: Measuring Quality in Lossy Compression

For lossless compression, the quality metric is binary: the output is either a perfect replica of the input, or it is not. With lossy compression, the concept of quality becomes much more complex and central to the evaluation. Since we are intentionally discarding data, the critical question is: "How much damage has been done, and is it noticeable?"

Subjective Quality Assessment

The ultimate judge of the quality of a compressed image or audio file is the human sensory system. Subjective quality assessment relies on human observers to rate the perceived quality.

The most common method is the Mean Opinion Score (MOS) test. In a controlled environment, a group of test subjects is shown or played the compressed media alongside the original. They are then asked to rate the quality of the compressed version on a scale, typically from 1 (bad) to 5 (excellent). The scores are then averaged to produce a single MOS value.

While MOS is considered the "gold standard" because it directly measures human perception, it has significant drawbacks. It is extremely expensive, time-consuming, and the results can vary depending on the observers and the test conditions. It cannot be automated as part of a development or testing pipeline.

Objective Quality Assessment

To overcome the limitations of subjective tests, engineers use objective quality assessment, which employs mathematical formulas to compare the compressed file to the original.

A foundational metric is the Mean Squared Error (MSE), which calculates the average of the squared differences between the original and compressed data points (e.g., pixel values). While useful, MSE is not very intuitive.

The most widely used objective metric for images and video is the Peak Signal-to-Noise Ratio (PSNR). PSNR is derived from MSE but is presented on a logarithmic decibel (dB) scale, which aligns better with how humans perceive changes in quality.

Here, $MAX_I$ is the maximum possible value for a data point (e.g., 255 for an 8-bit pixel). A higher PSNR value indicates a better quality image, meaning the "signal" (original image) is much stronger than the "noise" (the compression error). A PSNR above 35-40 dB typically indicates that the compressed image is visually indistinguishable from the original to most viewers. However, PSNR has its limits; it measures absolute mathematical error and does not always perfectly capture perceived visual quality. An algorithm might achieve a high PSNR but produce artifacts that are more jarring to the human eye than another algorithm with a slightly lower PSNR. This has led to the development of more advanced metrics like SSIM (Structural Similarity Index Measure) that try to better model human perception.

The Visual Price of Lossy Compression: Compression Artifacts

When lossy compression is pushed too far (i.e., a very high compression ratio is chosen), the discarded information becomes noticeable to the user. These perceptible distortions are known as compression artifacts.

Adjust the slider to see how JPEG settings balance compression ratio against the visibility of blocking, ringing, and blur.

Common artifacts include:

• Blocking (Blockiness): The most common artifact in DCT-based compression (like JPEG). The image appears to be divided into a grid of 8x8 pixel blocks, as the compression for each block is handled independently and the discontinuities at the block edges become visible.
• Ringing (or Mosquito Noise): Appears as ghostly halos or noise around sharp edges and contours in an image. It is caused by the inaccurate representation of high-frequency components that define those edges.
• Blurring: The loss of fine details and textures, making the image appear soft or out of focus. This is a direct result of aggressive quantization of high-frequency coefficients.
• Color Banding (Posterization): Smooth gradients of color are replaced with distinct bands or steps of solid color. This happens when the number of available colors is drastically reduced. This is a common artifact in GIF files with too few colors in the palette.

Explore how shrinking the palette for an indexed image quickly introduces noticeable colour banding.

The Compression Trilemma: You Can't Have It All

Ultimately, evaluating compression algorithms is about balancing a three-way trade-off between compression ratio, speed, and quality. It is generally impossible to maximize all three simultaneously. Improving one almost always comes at the expense of at least one of the others. The "best" algorithm is the one that strikes the right balance for a given application. For archival purposes, one might choose a very slow algorithm that provides the highest possible lossless compression ratio. For a live video chat, speed is paramount, so a balance is struck between an acceptable quality and a low enough compression ratio to be encoded in real-time. For professional photography, quality is king, and larger file sizes are an acceptable consequence.