Load Balancing

L4 vs L7 load balancers, algorithms, health checks, global traffic management, connection draining and the failure modes of load balancing itself.

What it is

A load balancer distributes requests across healthy backends to improve throughput, availability and latency. It also provides a stable entry point, TLS termination and a place to apply routing rules.

Layer Works on Can route by Examples
L4 (transport) TCP/UDP connections IP, port AWS NLB, Azure Load Balancer, GCP Network LB, HAProxy TCP mode
L7 (application) HTTP/gRPC requests Host, path, headers, cookies AWS ALB, Azure Application Gateway, GCP Application LB, Envoy, NGINX
Global (DNS / anycast) Users to regions Geography, latency, health Route 53, Azure Front Door/Traffic Manager, Google global LB, Cloudflare
Client-side Inside the caller Service registry data gRPC client LB, Spring Cloud LoadBalancer

Algorithms

Algorithm Good for Watch out
Round robin Homogeneous, short requests Ignores current load
Least connections / least outstanding requests Variable request durations Needs connection tracking
Weighted Mixed instance sizes, canary splits Weights must be maintained
Consistent hashing Cache affinity, sticky partitions Uneven load with hot keys
Power of two choices Large fleets; pick the less loaded of two random backends Simple and very effective
flowchart TB
  U[Users] --> DNS[Global DNS / anycast]
  DNS --> R1[Region A L7 LB]
  DNS --> R2[Region B L7 LB]
  R1 --> AZ1a[AZ-a targets]
  R1 --> AZ1b[AZ-b targets]
  R1 --> AZ1c[AZ-c targets]
  R2 --> AZ2a[AZ-a targets]
  R2 --> AZ2b[AZ-b targets]

Operational features

  • Health checks: active (probe endpoints) and passive (observe errors). Health endpoints should check what the instance needs to serve traffic, but avoid failing every instance because a shared dependency is down.
  • Connection draining / deregistration delay for zero-downtime deploys.
  • TLS termination and re-encryption to backends where required.
  • Sticky sessions: avoid where possible; externalize session state instead.
  • Cross-zone load balancing and zonal shift to evacuate a bad AZ.

Real-world example

A video platform uses anycast global load balancing to send users to the nearest healthy region, L7 load balancers per region routing /api/* to API services and /upload/* to upload services, and gRPC client-side load balancing between internal services to avoid an extra network hop.

Advantages and disadvantages

Advantages Disadvantages
Horizontal scaling and failover Can itself be a bottleneck or SPOF if self-managed
Central TLS and routing Extra hop latency
Enables blue/green and canary Misconfigured health checks cause mass outages

Architect-level considerations

[!ARCHITECT] Use managed, multi-AZ load balancers. Separate liveness (restart me) from readiness (send me traffic) checks. Plan how traffic shifts between regions (DNS TTLs vs anycast) and what happens to in-flight sessions.

Interview questions and answers

L4 or L7 for gRPC services?

L7 (or client-side), because gRPC multiplexes many requests over one long-lived HTTP/2 connection. An L4 balancer would pin all of a client's requests to one backend.

How do you do zero-downtime deployment behind a load balancer?

Rolling or blue/green: bring up new instances, wait for readiness, shift traffic gradually, drain connections from old instances (deregistration delay), and roll back automatically on error-rate alarms.

Why can DNS-based failover be slow?

Clients and resolvers cache DNS records beyond the TTL. Anycast or a global proxy-based load balancer switches faster because the IP stays the same.