Load Balancing
L4 vs L7 load balancers, algorithms, health checks, global traffic management, connection draining and the failure modes of load balancing itself.
What it is
A load balancer distributes requests across healthy backends to improve throughput, availability and latency. It also provides a stable entry point, TLS termination and a place to apply routing rules.
| Layer | Works on | Can route by | Examples |
|---|---|---|---|
| L4 (transport) | TCP/UDP connections | IP, port | AWS NLB, Azure Load Balancer, GCP Network LB, HAProxy TCP mode |
| L7 (application) | HTTP/gRPC requests | Host, path, headers, cookies | AWS ALB, Azure Application Gateway, GCP Application LB, Envoy, NGINX |
| Global (DNS / anycast) | Users to regions | Geography, latency, health | Route 53, Azure Front Door/Traffic Manager, Google global LB, Cloudflare |
| Client-side | Inside the caller | Service registry data | gRPC client LB, Spring Cloud LoadBalancer |
Algorithms
| Algorithm | Good for | Watch out |
|---|---|---|
| Round robin | Homogeneous, short requests | Ignores current load |
| Least connections / least outstanding requests | Variable request durations | Needs connection tracking |
| Weighted | Mixed instance sizes, canary splits | Weights must be maintained |
| Consistent hashing | Cache affinity, sticky partitions | Uneven load with hot keys |
| Power of two choices | Large fleets; pick the less loaded of two random backends | Simple and very effective |
flowchart TB U[Users] --> DNS[Global DNS / anycast] DNS --> R1[Region A L7 LB] DNS --> R2[Region B L7 LB] R1 --> AZ1a[AZ-a targets] R1 --> AZ1b[AZ-b targets] R1 --> AZ1c[AZ-c targets] R2 --> AZ2a[AZ-a targets] R2 --> AZ2b[AZ-b targets]
Operational features
- Health checks: active (probe endpoints) and passive (observe errors). Health endpoints should check what the instance needs to serve traffic, but avoid failing every instance because a shared dependency is down.
- Connection draining / deregistration delay for zero-downtime deploys.
- TLS termination and re-encryption to backends where required.
- Sticky sessions: avoid where possible; externalize session state instead.
- Cross-zone load balancing and zonal shift to evacuate a bad AZ.
Real-world example
A video platform uses anycast global load balancing to send users to the nearest healthy region, L7 load balancers per region routing /api/* to API services and /upload/* to upload services, and gRPC client-side load balancing between internal services to avoid an extra network hop.
Advantages and disadvantages
| Advantages | Disadvantages |
|---|---|
| Horizontal scaling and failover | Can itself be a bottleneck or SPOF if self-managed |
| Central TLS and routing | Extra hop latency |
| Enables blue/green and canary | Misconfigured health checks cause mass outages |
Architect-level considerations
[!ARCHITECT] Use managed, multi-AZ load balancers. Separate liveness (restart me) from readiness (send me traffic) checks. Plan how traffic shifts between regions (DNS TTLs vs anycast) and what happens to in-flight sessions.
Interview questions and answers
L4 or L7 for gRPC services?
L7 (or client-side), because gRPC multiplexes many requests over one long-lived HTTP/2 connection. An L4 balancer would pin all of a client's requests to one backend.
How do you do zero-downtime deployment behind a load balancer?
Rolling or blue/green: bring up new instances, wait for readiness, shift traffic gradually, drain connections from old instances (deregistration delay), and roll back automatically on error-rate alarms.
Why can DNS-based failover be slow?
Clients and resolvers cache DNS records beyond the TTL. Anycast or a global proxy-based load balancer switches faster because the IP stays the same.