Why We Needed a New HTTP: The Limitations of HTTP/1.1
For nearly two decades, HTTP/1.1 provided the foundation for the growth of the World Wide Web. It introduced persistent connections, allowing multiple requests and responses to be sent over a single TCP connection, which was an important change from its predecessor, HTTP/1.0. However, as websites evolved from simple text documents into complex applications with dozens of scripts, stylesheets, and images, HTTP/1.1 limitations began to restrict performance.
HTTP/1.0 allowed only one outstanding request per connection. HTTP/1.1 added pipelining, but clients commonly opened several connections to increase concurrency. Those connections also competed for congestion control and socket resources.
One HTTP/1.1 limitation is known as . A persistent connection did not provide multiplexing. Pipelining allowed several requests to be sent before responses arrived, but responses still had to preserve order. A slow request, perhaps for a large image or a slow API endpoint, could therefore delay later responses and page rendering.
Implementations used workarounds such as multiple TCP connections per hostname (historical browsers commonly used up to 6 connections per hostname), domain sharding, and file concatenation. These techniques added complexity and overhead. Over time, it became clear that the protocol needed a redesign for the demands of the modern web. This led to HTTP/2, which was largely based on Google's experimental SPDY protocol and standardized in 2015.
Core Concept 1: The Binary Framing Layer
One of the main changes introduced in HTTP/2 is the . HTTP/1.1 uses a textual message syntax. Request lines and fields use text separated by line endings, while message content can carry arbitrary octets. This format helps debugging but increases parsing complexity.
HTTP/2 divides communication into smaller units called frames and encodes them in binary form. This change reduces parsing overhead and supports concurrent streams:
- Efficiency: Binary framing reduces parsing overhead.
- Unambiguous structure: Frame fields reduce variation in textual parsing.
- Stream handling: Frames from multiple streams can be interleaved on one connection.
The Structure of an HTTP/2 Frame
Each frame in HTTP/2 has a well-defined structure. It consists of a fixed 9-byte header followed by a variable-length payload. The header combines the payload length, type, flags, a reserved bit, and the stream identifier. The receiver advertises the maximum payload size with SETTINGS_MAX_FRAME_SIZE.
- Length (
24bits) - An unsigned integer indicating the length of the frame payload in bytes.
- Type (
8bits) - Defines the type of the frame. This determines how the payload should be interpreted.
- Flags (
8bits) - A set of eight single-bit flags specific to the frame type, used to modify the frame's behavior.
- Reserved field (
1bit) - A field without defined semantics that senders set to zero.
- Stream Identifier (
31bits) - A unique identifier for the stream to which this frame belongs and that is used for multiplexing.
- Frame Payload (Variable)
- The actual content of the frame, whose structure is determined by the frame's Type field.
Common Frame Types
Some of the most important frame types include:
- DATA: Carries the message body for a request or response.
- HEADERS: Contains the HTTP headers for a request or response.
- PRIORITY: A legacy frame for stream priority; its older signaling scheme is deprecated.
- SETTINGS: Conveys configuration parameters for the connection.
- PUSH_PROMISE: Used by the server to initiate a server push.
Core Concept 2: Streams and Multiplexing
The binary framing layer enables a core HTTP/2 feature: . It removes blocking caused by the HTTP/1.1 exchange model, but a lost TCP packet can still stall all streams.
Streams: The Virtual Channels
Within a single HTTP/2 connection, the client and server can establish multiple independent, bidirectional sequences of frames called streams. One TCP connection carries frames for all streams, which can represent an HTML document, a CSS file, or an image and can be processed concurrently.
- Each stream is assigned a unique Stream Identifier.
- A request and its corresponding response occur within the same stream.
- Client-initiated streams always have odd-numbered IDs, while server-initiated streams (used for Server Push) have even-numbered IDs.
- A single connection can handle hundreds or thousands of concurrent streams.
Stream 0 is reserved for connection-level frames such as SETTINGS and GOAWAY. DATA frames are subject to per-stream and connection-level flow control, so a sender must respect the receiver's advertised windows.
How Multiplexing Solves HOL Blocking
Because communication is broken down into small, independent frames, each tagged with its stream ID, the client and server can interleave frames from multiple streams on the same connection. If a response for a large image (on Stream 5) is delayed, the server can still send frames for the CSS file (on Stream 3) and the JavaScript file (on Stream 7) at the HTTP layer. A lost TCP packet can still delay frame delivery for every stream until retransmission.
This parallel processing means that a slow HTTP response does not have to block faster responses. It reduces the need for workarounds such as multiple connections and can lower resource use while improving network utilization.
Core Concept 3: Server Push
Traditionally, HTTP communication is strictly initiated by the client. The browser requests an HTML file, parses it, finds references to CSS and JavaScript files, and then sends new requests for those resources. This creates a waterfall of requests, with inherent delays between each step.
HTTP/2 introduces an optional mechanism called . A server can anticipate which resources a client may need and send them before an explicit request.
How Server Push Works
- Client Request: The browser sends a normal request for a resource, for example,
index.html. - Server Analysis: The server receives the request. It knows that any browser requesting
index.htmlwill immediately also needstyle.cssandapp.jsto render the page. - Server Pushes Resources: Before sending the response for
index.html, the server sends specialPUSH_PROMISEframes to the client. These frames indicate thatstyle.cssandapp.jswill follow, so the client need not request them. The server then sends the data for these resources on new, server-initiated streams. - Original Response: The server then sends the original response for
index.html. - Client Receives and Uses: The browser receives all three resources nearly simultaneously. When it starts parsing the HTML and finds the links to the CSS and JS files, it can use the pushed responses when their cache semantics allow it.
Server Push can reduce page load time by avoiding a separate request for a predicted resource. It requires control of the client cache state: pushing an already cached resource increases network use. Deployments often use preload hints instead because their caching behavior is easier to control.
A pushed response is associated with a synthetic request and is subject to HTTP response semantics, including cacheability. The client can cancel a push when the resource is unnecessary.
Other Major Improvements in HTTP/2
Beyond the three core concepts, HTTP/2 introduced several improvements that affect performance.
- Header Compression (HPACK)
- In HTTP/1.1, field names and values use a textual representation and are often repeated across requests and responses. For example, a browser might send identical
User-AgentandAcceptfields many times for one page load. HTTP/2 introduces HPACK, which uses static and dynamic tables to encode repeated fields more compactly. The compression state is maintained per connection, and a field block can span HEADERS and CONTINUATION frames. This can reduce transmitted data, especially on mobile networks. - Stream Prioritization
- With multiplexing, a browser can make many requests at once, but resources can have different rendering priorities. A render-blocking CSS file can take precedence over an image at the bottom of the page. HTTP/2 permits priority information for streams; the older priority tree is deprecated in the current HTTP/2 specification, and a server applies its own scheduling policy to available signals. Priority does not change stream semantics or guarantee a particular delivery order.
- One Connection Per Origin
- Multiplexing reduces the need for multiple TCP connections to one origin. A client can often reuse one connection, reducing handshake and socket overhead, but connection reuse still depends on origin, certificate, configuration, and network conditions.
- Secure by Default (in practice)
- The HTTP/2 specification permits cleartext operation, but an
httpsURI uses TLS and negotiates theh2protocol identifier with ALPN (Application-Layer Protocol Negotiation).