Home / System Design / Concepts / API Gateway CONCEPT SHEET · Concepts
API Gateway
One-liner: Single entry point in front of services that handles routing, auth, rate limiting, and protocol translation so services don't.
Article Concepts © Arthur Bernier Jr 445 words 1 code block
Say this#
SAY THIS
Edge / ingress — where traffic enters; the gateway sits here behind the LB.Cross-cutting concerns — authn/authz, rate limiting, TLS termination, logging, CORS.Routing / path-based routing — /users/* → user-service.Protocol translation — REST/JSON in, gRPC out; WebSocket upgrade.Request aggregation / BFF — Backend-for-Frontend: one gateway per client type composing multiple calls.Auth offload — validate JWT / API key once at the edge.Circuit breaker / retries / timeouts — resilience policies applied at the gateway.Canary / traffic splitting — weighted routing to new versions.Examples — Kong, AWS API Gateway, Apigee, Nginx, Envoy, Traefik, Zuul, Spring Cloud Gateway.
How it works#
TEXT Copy
client -> DNS -> [L4/L7 LB] -> [API Gateway] -> service A |---------> service B |---------> service C
TLS terminates at LB or gateway.
Gateway authenticates (JWT verify, API key lookup), applies rate limit (Redis-backed counters).
Matches route → target service, rewrites path/headers, adds correlation ID.
Optional: transform payload, aggregate calls (BFF), cache GETs.
Returns response; emits metrics/logs/traces.
Gateway vs load balancer: LB distributes identical traffic; gateway understands the API and makes policy decisions. Gateway vs service mesh: gateway is north-south (edge), mesh (Istio/Linkerd sidecars) is east-west (service-to-service).
Trade-offs#
Choice Pro Con Single gateway One place for auth/limits/observability SPOF and bottleneck if not horizontally scaled; team coupling BFF per client Tailored payloads, mobile vs web More code to maintain Logic in gateway Services stay thin Gateway becomes a "smart pipe" monolith; deploy coupling Managed (AWS API GW) No ops, built-in throttling/auth Per-request cost, latency, vendor limits Self-hosted (Kong/Envoy) Control, performance You run it HA
Numbers#
Gateway hop typically adds ~1-5 ms (order of magnitude), more for managed cloud gateways.
AWS API Gateway default account throttle: 10,000 req/s with 5,000 burst (soft limit); 29 s integration timeout.
Nginx/Envoy single instance: tens of thousands of req/s on commodity hardware.
In the interview#
SAY THIS
Draw it right after the load balancer whenever there is more than one service or public clients: "Gateway handles auth, rate limiting, and routing."
Say: "Stateless gateway instances behind the LB, auto-scaled; rate-limit state in Redis."
Follow-up: "Isn't that a single point of failure?" → "Multiple instances, health-checked by the LB; config in a store like Consul/etcd; the LB itself is a redundant pair."
Follow-up: "Gateway vs service mesh?" → "Gateway is edge/north-south; mesh is internal/east-west with sidecars. Often both."
Creator
Written by Arthur Bernier Jr (Big Poppa Code). Arthur Bernier Jr — Big Poppa Code — is a software engineer, educator and entrepreneur: a former CTO at VC-backed startups and engineering lead at a billion-dollar company, mentor to 800+ engineers, now Senior Director of AI, Computer Programming and Cloud Computing at Per Scholas, and the host of the Deeper Than Code podcast. Everything on this page is his original teaching material.
bigpoppacode.io
License © Arthur Bernier Jr all rights reserved