Higher Resolutions and the Successor to H.264
H.264/AVC was widely used for video distribution, including Blu-ray Discs and internet services. As displays and cameras improved, resolutions above Full HD () became more common, including 4K () and 8K. More picture samples increase bitrate, decoder throughput, buffer memory and processing time.
A 4K frame has four times as many pixels as a 1080p frame, and an 8K frame has sixteen times as many. H.264 can encode such pictures with suitable profiles and levels, but the required bitrate can restrict streaming and broadcasting. A successor was needed to improve coding efficiency without simply multiplying the required bandwidth by four. Resolution also affects decoder memory and processing, so bitrate does not describe the complete cost. The same resolution can require different rates for film grain, fast motion, camera noise and scene changes. A codec comparison therefore needs a stated picture format, frame rate, quality measure and encoder configuration.
The Joint Collaborative Team on Video Coding (JCT-VC), formed by experts from ITU-T and ISO/IEC, developed a successor to H.264. The design target was similar perceptual quality at roughly half the bitrate, although the measured result depends on content, settings and the quality metric. The result, standardized in 2013, was , also known as H.265. HEVC separates decisions about prediction, transform and entropy coding, so an encoder can trade computation time against bitrate and quality. The decoder follows the syntax and reconstructed reference samples signalled by the bitstream.
The Main Change: From Macroblocks to Coding Trees
One of the main changes in H.265 is how a picture is partitioned. H.264 uses pixel macroblocks, while HEVC introduces the , with a maximum size of samples. The partition can follow scene complexity and affects prediction and transform decisions. The maximum size is signalled in the sequence parameters, so the decoder knows the tree limits before processing the picture. CTUs are units of syntax and processing rather than a promise that every region will use the maximum size. Flat regions may benefit from a shallow tree, while motion boundaries and fine texture usually lead to smaller coding regions.
The Coding Tree Unit (CTU)
In HEVC, the CTU size can be , or samples, according to the sequence parameters. A larger block can describe a broad, flat area such as a sky or wall efficiently, while detailed content usually requires further partitioning. The CTU size defines the picture structure; CU sizes result from recursively partitioning that area.
A CTU has a recursive quadtree structure. A complete region can remain one block or be split into four units, which can be split further down to a minimum CU size of . A transform can have a smaller size, including . Separating CU and TU lets the encoder choose coding and transform partitions for the content. CU defines the decision region, PU describes prediction, and TU defines the residual transform region; their boundaries need not be identical. The distinction matters when reading a bitstream: a CU carries coding decisions, a PU carries prediction decisions, and a TU carries transform coefficients. Confusing the minimum CU with the minimum transform assigns the wrong meaning to the block dimensions.
The Tree Branches: CU, PU and TU
Within the CTU, the encoder makes decisions at three partitioning levels, each with a different purpose:
- Coding Units (CUs): A CU results from the main quadtree partitioning of the CTU. The encoder chooses the depth according to the picture content. A flat region may be coded as one CU, while detailed texture may be divided into smaller CUs. CU size defines the area for subsequent prediction and transform decisions.
- Prediction Units (PUs): Within a CU, the encoder selects intra- or inter-prediction and partitions it into one or more using an allowed mode. For motion prediction, a large CU can use asymmetric PUs, such as two rectangles, to follow an object's shape and movement. This extends the fixed sub-macroblock options of H.264. Inter prediction uses motion compensation from reference pictures; the PU shape controls the region for which that motion description applies.
- Transform Units (TUs): After prediction, the residual, meaning the difference between the original and prediction, is transformed and quantized. The CU can be divided into a tree of . HEVC uses integer transforms up to , whereas H.264 uses sizes up to . A larger transform can concentrate residual energy in smooth regions, with additional computation and signalling cost. Quantization removes less visible precision from transform coefficients, and the entropy coder represents the remaining nonzero coefficients and their positions.
The CTU structure permits large blocks for simple areas and smaller blocks for details. Partitioning affects the number of decisions, signalling and residual data, so it is an important part of HEVC efficiency, but it is not the only one.
The interactive example compares H.264 macroblocks with the adaptive HEVC CTU quadtree for scenes of different complexity.
Prediction and Coding Tools
In addition to CTUs, H.265/HEVC defines prediction, filtering and entropy-coding tools that reduce residual data and signalling cost. Their gains interact: better prediction can produce a smaller residual, while a different partition can change both prediction cost and the number of syntax elements. No single tool determines the bitrate of a complete sequence.
1. Enhanced Intra-Prediction
For luma blocks, H.264 offered 9 intra-prediction modes. H.265 provides 35 modes for luma intra-prediction: DC, Planar and 33 angular modes. More directions can predict edges and textures more closely and may reduce the residual. The number refers to available luma modes; a block does not use all of them. The encoder selects a mode using prediction, signalling and quantization costs. The selected mode is signalled compactly and uses neighboring reconstructed samples, which keeps the prediction available at both encoder and decoder.
2. Motion Prediction: Merge and AMVP
In P and B pictures, H.265 improves motion-vector coding. Two modes are used for Prediction Units:
- Merge Mode: The encoder builds a candidate list from neighboring spatial and temporal blocks. When a candidate is suitable, it sends an index instead of a new motion vector or vector difference. The prediction residual can still be coded. The mode reduces signalling when the motion is consistent because the decoder derives the same vector from the candidate list.
- Advanced Motion Vector Prediction (AMVP): This mode derives a prediction from neighboring vectors and additionally codes the difference between the predicted and selected vector. It is useful when motion resembles that of neighboring blocks but is not identical.
3. Sample Adaptive Offset (SAO)
SAO is an in-loop reconstruction filter used alongside deblocking. It classifies samples by band or edge category and adds a signalled offset to the selected category. The operation corrects recurring quantization errors and affects both picture quality and the reconstructed samples used as references for later prediction. The stream carries the SAO parameters, so the decoder can apply the same categories and offsets without the source picture. SAO therefore belongs to the reconstructed reference path and affects samples used by later decoding.
Parallel Processing: Tiles and WPP
Dependencies between blocks restrict parallel encoding and decoding. HEVC defines tiles and Wavefront Parallel Processing (WPP) to distribute part of the work across cores while retaining required contexts. Parallel tools change scheduling and memory access; they do not remove the need to parse the bitstream in its defined order or to respect reference-picture dependencies.
- Tiles: The encoder divides a picture into rectangular regions. Tiles have independent entropy coding and restricted spatial dependencies, so they can be assigned to different cores. A player on an 8-core computer can process several tiles in parallel. Whether in-loop filters cross tile boundaries depends on the relevant flags, so a filter is not always limited to the tile interior. More tiles can increase signalling overhead and weaken prediction between neighboring regions. Tile dimensions and the number of tiles are signalled in picture-parameter information, and an implementation still needs enough work per tile to benefit from parallel scheduling.
- Wavefront Parallel Processing (WPP): WPP processes CTU rows in parallel with a delay needed to obtain context from the row above. The next row can start before the previous row finishes, creating a moving wavefront. This provides parallelism without tiles, although context dependencies limit it for short rows or small pictures. WPP can preserve more shared picture context than independent tiles, but available parallelism depends on row length and decoder scheduling overhead.
These tools help multi-core processors with 4K and 8K encoding and decoding. Real-time operation still depends on hardware support, implementation, settings and content complexity. Software encoders can select faster search settings, whereas hardware encoders often expose a smaller set of partition and prediction decisions. Those choices affect measured bitrate and quality.
The interactive visualisation shows how tile layouts and WPP scheduling translate into decoder parallelism.
Summary: Efficiency and Computational Cost
H.265/HEVC was designed to improve coding efficiency. Many comparisons report a 40-50% bitrate reduction relative to H.264 at similar perceptual quality, but the value depends on content, settings and measurement method. A lower bitrate can ease UHD transmission and reduce storage for video. It is not a fixed relation for every encoder, profile or sequence. The percentage describes a comparison at an equivalent quality point, not a guarantee that a source file becomes 40-50% smaller. Container overhead, audio, subtitles and the chosen rate-control mode also affect the delivered file.
Efficiency has a computational cost. Flexible partitioning and prediction tools increase the encoder's decisions for CU, PU and TU. An implementation can evaluate different partitions and the 35 intra-prediction modes. Depending on implementation, settings and content, encoding can require up to 10 times the computational power of H.264. Decoding is also more complex, although tiles and WPP can support parallel execution. The 10-times value describes the upper range of reported comparisons, not a universal overhead. Measurements should state whether they compare single-pass or multipass encoding, software or hardware, and which H.264 and HEVC presets were used. Without those conditions, a speed ratio is not portable between systems.
H.265/HEVC is intended for content including Ultra High Definition and remains widely deployed. Newer standards such as VVC (H.266) continue to improve coding efficiency, while codec selection depends on hardware, licensing, the ecosystem and service requirements. In practice, deployment also depends on decoder availability, streaming format support and the computational budget of target devices.