Idempotency

One-liner: An operation is idempotent if doing it N times has the same effect as once; it is what makes retries and at-least-once delivery safe.

Say this#

SAY THIS

Idempotent — f(f(x)) = f(x); SET x=5 yes, x += 1 no.
Idempotency key — client-generated unique ID per logical operation (Idempotency-Key header; Stripe popularized it).
At-least-once delivery — broker/retries may duplicate; the default in most queues.
At-most-once — fire and forget; may lose.
Exactly-once processing — at-least-once delivery + idempotent/deduplicating consumer; the achievable version.
Dedupe table / seen-set — store processed IDs with TTL (Redis SETNX, unique index in DB).
Natural idempotency — upserts, PUT with full state, conditional writes (IF version = n).
Conditional write / compare-and-set — optimistic concurrency: UPDATE ... WHERE version = 3.
Retry with exponential backoff and jitter — the client side of the contract.
Retry storm — synchronized retries amplify an outage; jitter and budgets prevent it.
Safe vs idempotent HTTP methods — GET/HEAD safe; GET/PUT/DELETE idempotent; POST/PATCH not by default.

How it works#

TEXT

client --POST /payments, Idempotency-Key: 7f3a--> serverserver: lookup key in Redis/DB   hit, in-progress  -> 409 or wait   hit, done         -> return stored response (same status/body)   miss              -> INSERT key (unique) ; do the work ; store response ; returnclient --POST /payments, Idempotency-Key: 7f3a--> server
server: lookup key in Redis/DB
   hit, in-progress  -> 409 or wait
   hit, done         -> return stored response (same status/body)
   miss              -> INSERT key (unique) ; do the work ; store response ; return

Client generates a UUID per intent (same key on retries of the same action; new key for a new action).
Server atomically claims the key (INSERT ... ON CONFLICT DO NOTHING or SETNX key value EX 86400).
Executes the side effect once; persists result keyed by the idempotency key (ideally in the same transaction).
Replays return the stored result. Keys expire after 24 h (Stripe) or a reasonable window.

Consumer-side (queues): processed_events(event_id PK) inserted in the same DB transaction as the business write; duplicate event → unique violation → ack and skip.

Trade-offs#

Choice	Pro	Con
Idempotency keys	Safe client retries, exactly-once semantics for POST	Storage for keys/responses; key scoping (per user)
Dedupe in Redis	Fast, TTL built in	Not transactional with the business write; risk if Redis loses data
Dedupe in DB (unique index)	Atomic with the write	DB load; table growth needs TTL/partition pruning
Natural idempotency (upsert, CAS)	No extra state	Not every operation can be expressed this way
Broker "exactly-once" (Kafka transactions)	Handles Kafka→Kafka pipelines	Doesn't cover external side effects (email, payment)

Numbers#

Stripe idempotency keys are honored for 24 hours.
Dedupe window should exceed the maximum retry horizon (broker redelivery + client backoff), typically hours to a day.
SQS FIFO dedupe window: 5 minutes based on MessageDeduplicationId.

Say this#

How it works#

Trade-offs#

Numbers#

In the interview#