Load Balancers

One-liner: Distribute traffic across healthy backends to scale horizontally and hide failures; L4 balances connections, L7 balances requests.

Say this#

SAY THIS

L4 (transport) — routes by IP/port, TCP/UDP; no payload inspection; very fast (AWS NLB, HAProxy TCP mode, IPVS/LVS, Maglev).
L7 (application) — understands HTTP/gRPC: path/host/header routing, TLS termination, cookies (AWS ALB, Nginx, HAProxy, Envoy, Traefik).
Round robin / weighted round robin — rotate; weights for heterogeneous nodes.
Least connections / least outstanding requests — pick the least busy; good for variable request cost.
Least response time — pick fastest recent responder.
IP hash / source hash — same client → same backend; crude stickiness.
Consistent hashing / ring hash / Maglev — key-based affinity (cache locality) with minimal remap.
Random / power of two choices (P2C) — pick 2 at random, take the less loaded; near-optimal cheaply (Envoy default).
Health checks — active (probe /healthz) and passive (observe failures); eject unhealthy.
Sticky sessions / session affinity — cookie-based pinning; avoid by externalizing sessions.
Connection draining / deregistration delay — finish in-flight requests before removing a node.
Global server load balancing (GSLB) — DNS/anycast across regions.
VIP / VRRP / keepalived — floating IP for an HA LB pair.
DSR (direct server return) — backend replies directly to client; L4 trick for huge egress.

How it works#

TEXT

DNS/anycast (GSLB) -> region   -> [L4 LB pair, VIP via VRRP]  -> [L7 LB / gateway x N] -> service instancesDNS/anycast (GSLB) -> region
   -> [L4 LB pair, VIP via VRRP]  -> [L7 LB / gateway x N] -> service instances

Client resolves DNS to a regional VIP (or anycast IP).
L4 LB spreads TCP connections to L7 LBs (or straight to services).
L7 LB terminates TLS, inspects request, applies routing (host/path), picks a backend via algorithm.
Health checks remove failing instances; connection draining on deploys.
LB itself is redundant: active-passive pair with a floating VIP, or active-active behind ECMP/anycast. Cloud LBs (ALB/NLB) are managed, multi-AZ by default.

Trade-offs#

Choice	Pro	Con
L4	Millions of conns, protocol-agnostic, low latency	No content routing, no per-request balancing (long-lived conns skew)
L7	Smart routing, TLS offload, retries, observability	More CPU, more latency, more config surface
Round robin	Simple	Ignores backend load and request cost
Least connections	Adapts to load	Needs connection tracking; herd to a fresh node
Consistent hashing	Cache affinity	Hot keys; uneven load
Sticky sessions	Legacy apps work	Breaks on scale-in; hot nodes; use Redis sessions instead
Hardware/managed LB	Turnkey HA	Cost, less control

Numbers#

Nginx/HAProxy on a modern box: tens of thousands of HTTP req/s per core order of magnitude; L4 (IPVS/NLB) handles millions of concurrent connections.
Typical health check: every 5-30 s, 2-3 consecutive failures to eject, 2 successes to re-add.
LB hop latency: sub-millisecond for L4, ~1 ms for L7.
Connection draining defaults: AWS ALB 300 s deregistration delay.

Say this#

How it works#

Trade-offs#

Numbers#

In the interview#