Scaling: Horizontal vs Vertical

One-liner: Vertical scaling buys a bigger box; horizontal adds boxes, which requires stateless services and shared-nothing design.

Say this#

SAY THIS

Vertical scaling (scale up) — more CPU/RAM/IOPS on one node; no code changes; hard ceiling.
Horizontal scaling (scale out) — more nodes behind a load balancer; near-linear if stateless.
Stateless service — no per-client state in process memory; any instance can serve any request.
Session externalization — sessions in Redis/Memcached or in a signed cookie/JWT, not on the server.
Shared-nothing architecture — nodes don't share memory or disk; coordinate only via network.
Auto-scaling — add/remove instances on CPU, request count, or queue depth (ASG, HPA, KEDA).
Scale cube (AKF) — X: clone instances; Y: split by function/service; Z: split by data (shard).
Elasticity — scale down too; pay for what you use.
Cold start / warm-up — new instance latency penalty (JIT, cache fill, connection pools).
Capacity planning / headroom — target utilization ~50-70% to absorb spikes and failures (N+1).
Stateful scaling — databases scale via replicas (reads) and shards (writes); harder.
Serverless — scale-to-zero functions (Lambda, Cloud Run); horizontal scaling fully managed.

How it works#

TEXT

Scale cube:  X-axis: [app][app][app] behind LB            (clone)  Y-axis: [users svc] [orders svc] [search svc] (functional split)  Z-axis: [shard 0] [shard 1] [shard 2]         (data split by key)Scale cube:
  X-axis: [app][app][app] behind LB            (clone)
  Y-axis: [users svc] [orders svc] [search svc] (functional split)
  Z-axis: [shard 0] [shard 1] [shard 2]         (data split by key)

Start vertical: bigger instance, tune, add indexes; cheapest engineering.
Make app tier stateless: sessions to Redis, uploads to S3, local caches tolerated as best-effort.
Put instances behind an LB; auto-scale on a leading indicator (requests per target, queue depth) not just CPU.
Scale reads with cache + replicas; scale writes with sharding (Z-axis) only when forced.
Pre-warm before predictable spikes; set min instances to avoid cold-start cliffs.

Trade-offs#

Choice	Pro	Con
Vertical	Zero architecture change, strong consistency stays simple	Ceiling, cost grows superlinearly, SPOF, downtime to resize
Horizontal	No ceiling in principle, fault tolerance, elastic	Needs statelessness, LB, distributed coordination, data layer becomes bottleneck
Sessions in Redis	Any node serves any user	Redis is a dependency; extra hop
Sessions in JWT cookie	No server state	Revocation hard; size on every request
Auto-scaling on CPU	Simple	Lags behind IO-bound or queue-driven load
Serverless	Scale-to-zero, no fleet ops	Cold starts, runtime limits, cost at steady high load

Numbers#

Largest cloud VMs today: hundreds of vCPUs and tens of TB RAM (e.g. AWS u-series up to 24-32 TB); vertical goes far for databases.
Stateless app tier: a modest node serves roughly thousands of req/s for simple endpoints; scale count = peak QPS / per-node capacity / 0.7 headroom, plus 1.
Auto-scaling reaction time: ~1-5 min for VMs, seconds for containers; design buffers (queues) for that gap.
Lambda cold start: ~100 ms to a few seconds depending on runtime and package size.

Say this#

How it works#

Trade-offs#

Numbers#

In the interview#