JPEG Compression

Step 1: Color space transformation

Most digital images are captured and initially stored in the RGB (Red, Green, Blue) color space, where each pixel's color is defined by the intensity of these three primary colors. While this is intuitive for displays, it is not ideal for compression. The first step in the JPEG process is to convert the image from RGB to a different color space, typically YCbCr.

This new space separates the image information into three components:

• Y (Luminance): This component represents the brightness or grayscale information of the image. It is what you would see if you were looking at a black-and-white version of the photograph.
• Cb (Chrominance-blue): This component describes the difference between the blue component and luminance.
• Cr (Chrominance-red): This component describes the difference between the red component and luminance.

This separation is useful because vision is more sensitive to brightness (luma) detail than to color (chroma) detail. JPEG can therefore process the two types of components at different resolutions. JFIF files use the YCbCr color model and describe component sampling in the frame structure and headers. The sampling factors are stored per component so the decoder can rebuild the component planes.

Step 2: Chroma subsampling (downsampling)

After separating color from brightness, the algorithm can reduce the resolution of the Cb and Cr channels. Chroma subsampling stores color samples shared by small groups of pixels instead of one sample for every pixel.

This is often described using a three-part ratio, such as $4:4:4$, $4:2:2$, or $4:2:0$. For a block of 4 horizontal pixels:

• $4:4:4$: No subsampling. Every one of the 4 pixels has its own distinct Y, Cb, and Cr value. This preserves the maximum color quality.
• $4:2:2$: Horizontal subsampling. Each pair of two horizontal pixels shares the same Cb and Cr value, while each has its own Y value. The color resolution is halved horizontally.
• $4:2:0$: This is a common scheme in consumer video and JPEG files. Horizontal and vertical color resolution are both halved. A block of $2 \times 2$ pixels shares a single Cb and Cr value.

This step reduces the number of color samples before further coding. The notation describes relative component sampling, rather than a direct file-size ratio. Visibility of the change depends on the image, sampling scheme, and encoder parameters.

Step 3: Discrete cosine transform (DCT)

After color transformation and subsampling, each channel (Y, Cb, and Cr) is divided into $8 \times 8$ pixel blocks. Each block is processed by the Discrete Cosine Transform.

Instead of 64 pixel values, the DCT produces 64 frequency coefficients. The coefficients describe the contribution of cosine waves at different frequencies. The output is another $8 \times 8$ block whose values describe frequencies:

• The DC coefficient: The top-left value in the block ($F(0,0)$) is the DC coefficient. It corresponds to the average of the 64 samples after the scaling defined by the DCT convention, representing the block's overall level.
• The AC coefficients: The remaining 63 coefficients are the AC (Alternating Current) coefficients. They represent the details, textures, and edges within the block. Coefficients near the top-left represent low-frequency changes (gentle gradients), while coefficients towards the bottom-right represent high-frequency changes (fine details and sharp edges). Their order in the block determines the order used by the later zig-zag scan.

In a typical photographic $8 \times 8$ block, much of the energy is concentrated in the DC coefficient and a few low-frequency AC coefficients. High-frequency coefficients often have small values, which helps quantization and coding. The DCT itself is reversible in the mathematical model. Implementations can introduce small rounding errors, while the main information loss comes in the following stage.

Step 4: Quantization, the lossy step

Quantization permanently simplifies information. Each of the 64 coefficients in the DCT block is divided by a corresponding value from a 64-element quantization table, and the result is rounded to an integer. The table is transmitted or selected according to the JPEG stream, so the decoder can apply the same values during dequantization.

This table follows psycho-visual principles and image characteristics.

• The value in the top-left of the quantization table (for the DC coefficient) is typically small, preserving the block's average component level with high precision.
• Values get progressively larger for higher-frequency coefficients. This means fine details (high-frequency AC coefficients) are divided by large numbers, which causes their resulting values to be rounded to zero or very small integers.

In implementations such as IJG or libjpeg, an encoder's "quality" parameter may use a scale from 1 to 100 and scale the quantization table. A high setting such as 95 uses smaller values, preserves more detail, and produces a larger file. A low setting such as 20 uses larger values, simplifies more detail, and produces a smaller file. The 1 to 100 scale is an encoder convention rather than part of the JPEG standard. For coefficient $F(u,v)$ and table entry $Q(u,v)$, the encoder records, in simplified form, $round(F(u,v)/Q(u,v))$. After this step, many of the 63 AC coefficients can become zero.

Step 5: Entropy coding

After quantization, the coefficient block contains many zeros. The final step arranges and encodes the data with lossless methods.

1. Zig-zag scan: The 64 quantized coefficients are read out in a zig-zag pattern, starting from the DC coefficient in the top-left corner and moving towards the bottom-right. The purpose of this scan is to group the many zero-valued high-frequency coefficients into long, consecutive runs.
2. Run-length encoding (RLE): The long runs of zeros are then efficiently encoded. Instead of writing "0, 0, 0, 0, 0", RLE stores a code that means "five zeros."
3. Huffman coding: In the baseline process, the stream of RLE codes and non-zero AC coefficients, together with separately coded DC coefficients, is compressed with Huffman codes. DC coefficients are commonly coded as differences from the previous block, and an end-of-block marker describes remaining zeros. The JPEG standard also defines arithmetic-coded processes. Shorter codes for frequent values pack the data into the JPEG bitstream.