Why video compression is necessary
Static-image compression reduces data within one image. Video is a sequence of images displayed at a defined rate, so a codec must represent changes over time as well as spatial information. The frame count, resolution, and sample format make raw video data grow quickly. Frame rate also determines how much data must be processed every second.
Consider the raw size of a short, uncompressed clip. This example assumes a Full HD image ( pixels) with 24-bit RGB, meaning 8 bits for each red, green, and blue channel. The samples are stored without chroma subsampling. One frame requires:
Using decimal units, this is about per frame.
At 30 frames per second, one second of uncompressed video would require:
Under these assumptions, one minute would exceed 11 gigabytes and a two-hour movie would exceed a terabyte. The calculation describes uncompressed RGB, so it is not a codec bitrate. Practical codecs often convert RGB to YCbCr and subsample chroma, so their size also depends on the sample format. makes it practical to transmit and store material in services such as Netflix, Zoom, and smartphone sharing applications.
Video compression algorithms use two basic types of redundancy:
- Spatial redundancy (intra-frame): This concerns similar samples within one frame, as in a still image. Examples include areas of sky or a wall with a similar color.
- Temporal redundancy (inter-frame): This results from similarities between successive frames. During motion, part of the background remains still or changes predictably, so the encoder can describe the change with references and residual data.
Intra-frame and inter-frame compression in video codecs
A modern video codec can combine two strategies. One codes a picture without references to other frames, while the other uses similarities between pictures over time. A codec can be software or hardware responsible for encoding and decoding. Both strategies finish by entropy-coding the elements that remain after prediction or quantization.
Intra-frame compression for one picture
addresses spatial redundancy. The frame is coded without references to preceding or succeeding frames. The outline resembles JPEG, while specific video standards use their own transforms and data formats:
- The frame is divided into small blocks of pixels (e.g., ).
- Each block is processed with a DCT-like transform or an integer transform specified by the standard. The transform converts spatial values into coefficients and separates slowly changing components from fine details.
- The coefficients are then . Less important details are discarded or represented with less precision, which reduces data and can reduce quality.
- The resulting quantized coefficients are finally compressed losslessly using techniques like Huffman coding.
Frames coded only with this method are called I-frames (Intra-frames). They can serve as access points and references for later prediction. After the transform, coefficients are usually entropy-coded to reduce signaling overhead.
Inter-frame compression and motion prediction
addresses temporal redundancy. The encoder predicts the current frame from one or more decoded reference frames. B-frames can use pictures on both sides of the display order.
The core technique is motion compensation. An outline of the process is as follows.
- The encoder divides the current frame into blocks. Older standards often call such blocks macroblocks, for example pixels.
- For each macroblock in the current frame, the encoder searches a reference frame to find the block of pixels that is the closest match. Depending on the standard, a motion vector can represent a fractional-pixel displacement.
- Instead of encoding every macroblock pixel, the encoder records a . In a simplified example, it means taking a block from position (x, y) in a reference frame and using it at position (x', y') in the current frame.
- The predicted block can differ from the source image. The encoder calculates the difference between the actual block and its motion-compensated prediction. This difference, called the residual, is transformed, quantized, and coded. With a good match, it contains less information than the complete block.
Frames that are encoded using this predictive method are called P-frames (Predicted frames) and B-frames (Bi-directionally predicted frames).
I-, P-, and B-frame types
Inter-frame codecs often combine three frame types. Their arrangement affects bitrate, quality, access points, and decoder requirements. Names and detailed capabilities depend on the standard.
I-frames (Intra-frames or Keyframes)
I-frames use intra-frame coding and do not require other frames for decoding. They resemble complete pictures embedded in the stream, while their format and compression level are defined by the codec.
Role: They can provide access points for starting playback, seeking, and recovery after errors. A decoder still needs the applicable parameter sets and headers. Their size depends on the scene and settings, and is often larger than that of predictive frames.
P-frames (Predicted frames)
P-frames use motion compensation and one or more earlier reference pictures, depending on the codec. They contain a prediction description and residual data, so they are often smaller than I-frames.
Role: They carry changes between access points and limit the amount of coded data. Their dependencies can extend recovery after a lost packet.
B-frames (Bi-directionally predicted frames)
B-frames can use references from the past and future relative to display time. Two-sided prediction often improves the match and reduces residual data, while the result depends on the scene and codec settings.
Role: B-frames fill positions between reference pictures. They often reduce bitrate, while future references require buffering and a decoding order different from display order. The resulting delay depends on the GOP structure and implementation.
M-JPEG and inter-frame codecs
The compression standard affects file size, quality, latency, and editing. M-JPEG codes frames independently, while video standards such as MPEG-2, H.264, and H.265 also use inter-frame prediction.
M-JPEG (Motion JPEG): independent JPEG images
M-JPEG codes video as a series of independent JPEG images sent in sequence. Each frame can be decoded without a reference to neighboring frames. RTP/JPEG carries such independent JPEG frames with timestamps and fragmentation fields for network transport.
- Method: It uses only intra-frame compression. Each frame is an independent JPEG image, so the format does not use inter-frame prediction.
- Pros: The encoding and decoding path is simple. Frame-accurate editing is easier because the pictures are independent. The codec adds no delay from frame reordering, and corruption of one picture usually does not depend on earlier pictures.
- Cons: At a similar visual quality, files are usually larger than with codecs that use inter-frame prediction. This results from ignoring similarities between neighboring frames.
- Use Cases: Professional video editing workflows (as an intermediate format), some medical imaging systems, and high-end security cameras where the integrity of every individual frame is prioritized over file size.
Predictive codecs, including MPEG-2, H.264, and H.265
MPEG standards and H.264 and H.265 describe codecs that use intra-frame and inter-frame prediction. H.264 and H.265 are ITU-T and ISO/IEC standards with codec-specific tools. Motion compensation describes similarities between pictures, while the available tools depend on the standard.
- Method: Uses a mix of intra-frame compression (for I-frames) and inter-frame compression (for P- and B-frames) to remove both spatial and temporal redundancy. The encoder also signals prediction modes, motion data, transforms, and quantization parameters. The decoder applies the same syntax in the specified order.
- Pros: Inter-frame prediction can substantially reduce bitrate at a selected quality. This makes these codecs suitable for streaming, broadcast, and storage on media such as Blu-ray discs. Reference pictures can describe motion without repeating unchanged regions.
- Cons: Encoding and decoding require more computation. Editing is harder, and cuts without re-encoding are often made at access points such as IDR frames. An error in a reference frame can affect later pictures until the next access point, depending on error-resilience tools and transport.
- Use Cases: Common uses include online streaming, digital television, video conferencing, Blu-ray and DVD discs, and smartphone video recording. The codec choice depends on quality, bitrate, hardware requirements, access points, and decoder support.