Walkthrough: Chat System (WhatsApp / Slack / Messenger) · System Design

1. Requirements#

Functional

1:1 and group chat (groups up to ~500 members); text plus media attachments.
Real-time delivery when online; store-and-forward when offline; multi-device sync.
Message ordering within a conversation; sent/delivered/read receipts; online presence.

Non-functional

Message delivery latency < 200 ms when both online; durable (never lose an acked message).
50M DAU; 99.99% availability; messages persisted and retrievable (history, search optional).
End-to-end encryption optional (out of scope unless asked; affects server-side search).

2. Estimates#

Messages: 50M DAU × 40 msgs/day = 2B/day ÷ 86,400 ≈ 23k msg/s (peak ~5x ≈ 115k/s).
Storage: ~100 B text + ~100 B metadata = 200 B × 2B/day ≈ 400 GB/day ≈ 146 TB/year (text only). Media (10% of messages × 200 KB) ≈ 40 TB/day → S3 + CDN, lifecycle to cold storage.
Concurrent connections: assume 20% of DAU online at peak = 10M WebSocket connections. At ~100k-200k per gateway node → 50-100 gateway nodes (plus headroom).
Fan-out: group of 500 → one send = 500 deliveries; assume avg group message reaches 10 recipients → ~230k deliveries/s peak.
Bandwidth: 115k msg/s × 1 KB on the wire ≈ 115 MB/s; fine.

3. API sketch#

TEXT

WebSocket wss://chat.example.com/ws   (auth: token on connect)  client -> server:  { type:"send", clientMsgId, conversationId, text, mediaKey? }  server -> client:  { type:"ack", clientMsgId, messageId, serverTs }  server -> client:  { type:"message", messageId, conversationId, senderId, text, ts }  client -> server:  { type:"delivered"|"read", conversationId, messageId }  both: ping/pong every 30 sHTTP:  GET  /v1/conversations/{id}/messages?before=<messageId>&limit=50   (history / catch-up)  POST /v1/conversations        { memberIds[] }  POST /v1/media/upload-url     -> presigned URL  GET  /v1/users/{id}/presenceWebSocket wss://chat.example.com/ws   (auth: token on connect)
  client -> server:  { type:"send", clientMsgId, conversationId, text, mediaKey? }
  server -> client:  { type:"ack", clientMsgId, messageId, serverTs }
  server -> client:  { type:"message", messageId, conversationId, senderId, text, ts }
  client -> server:  { type:"delivered"|"read", conversationId, messageId }
  both: ping/pong every 30 s
HTTP:
  GET  /v1/conversations/{id}/messages?before=<messageId>&limit=50   (history / catch-up)
  POST /v1/conversations        { memberIds[] }
  POST /v1/media/upload-url     -> presigned URL
  GET  /v1/users/{id}/presence

4. Data model#

TEXT

messages  (Cassandra / HBase / DynamoDB; write-heavy, time-ordered reads)  PARTITION KEY conversation_id, CLUSTERING message_id DESC   -- message_id = Snowflake (time-ordered)  columns: sender_id, text, media_key, created_atconversations (Postgres): conversation_id, type(1:1|group), created_atmembers (Postgres/Cassandra): PK(conversation_id, user_id), joined_at, last_read_message_iduser_conversations (Cassandra): PK(user_id), CLUSTERING last_message_ts DESC  -- inbox listRedis:  presence:{user_id} -> {status, last_seen}  TTL 60 s (heartbeat refresh)        session:{user_id}  -> {gateway_node_id, device_ids[]}   TTL        unread:{user_id}:{conversation_id} countermessages  (Cassandra / HBase / DynamoDB; write-heavy, time-ordered reads)
  PARTITION KEY conversation_id, CLUSTERING message_id DESC   -- message_id = Snowflake (time-ordered)
  columns: sender_id, text, media_key, created_at
conversations (Postgres): conversation_id, type(1:1|group), created_at
members (Postgres/Cassandra): PK(conversation_id, user_id), joined_at, last_read_message_id
user_conversations (Cassandra): PK(user_id), CLUSTERING last_message_ts DESC  -- inbox list
Redis:  presence:{user_id} -> {status, last_seen}  TTL 60 s (heartbeat refresh)
        session:{user_id}  -> {gateway_node_id, device_ids[]}   TTL
        unread:{user_id}:{conversation_id} counter

5. High-level design#

TEXT

clients <--wss--> [L4 LB] --> [Chat gateway nodes x N]  (hold sockets, auth, heartbeats)                                     |  register user->node in Redis (session registry)                                     v                               [Chat service]  -> messages DB (Cassandra)                                     |          -> Kafka "message_sent" (fan-out, notifications, analytics)                                     v                          [Redis Pub/Sub or Kafka per-node topics]  -> gateway node holding recipient -> client                                     |                      offline?  -> [Notification service] -> APNs / FCM                                     |[Presence service] <- heartbeats; publishes status to friends' gateways[Media: presigned upload -> S3 -> CDN]     [History API -> Cassandra]clients <--wss--> [L4 LB] --> [Chat gateway nodes x N]  (hold sockets, auth, heartbeats)
                                     |  register user->node in Redis (session registry)
                                     v
                               [Chat service]  -> messages DB (Cassandra)
                                     |          -> Kafka "message_sent" (fan-out, notifications, analytics)
                                     v
                          [Redis Pub/Sub or Kafka per-node topics]  -> gateway node holding recipient -> client
                                     |
                      offline?  -> [Notification service] -> APNs / FCM
                                     |
[Presence service] <- heartbeats; publishes status to friends' gateways
[Media: presigned upload -> S3 -> CDN]     [History API -> Cassandra]

Send flow: client sends over WS → gateway → chat service assigns Snowflake message_id, writes to Cassandra, acks sender → looks up recipients' gateway nodes → publishes to those nodes → they push frames; offline recipients get a push notification and fetch on reconnect.

6. Deep dives#

Connection layer and routing

Stateful WebSocket gateways behind an L4 LB (connections stay put). Session registry in Redis maps user_id → node_id; messages are routed by publishing to a per-node channel or per-user channel the node subscribes to. Node death → clients reconnect with backoff; registry entries expire via TTL.

Ordering and IDs

Per-conversation ordering by message_id (Snowflake, time-ordered, generated server-side in one place per conversation or accept ms-level ties broken by ID). Clients dedupe by clientMsgId for idempotent resends; server stores clientMsgId briefly to return the same messageId on retry.

Storage choice

Cassandra/HBase: write-optimized LSM, partition by conversation_id, clustering by message_id DESC → "last 50 messages" is one partition range read. Wide partitions for busy groups → bucket by month: PARTITION (conversation_id, month).
Postgres for the relational bits (users, membership, groups) where joins and constraints matter.

Delivery semantics and sync

At-least-once from server to client; client-side dedupe by messageId. Receipts (delivered, read) are small events updating last_read_message_id; fan them back to the sender's devices. Multi-device: each device tracks its own cursor; on reconnect, GET messages?after=<last_seen_id>.

7. Bottlenecks & scaling#

Gateway fleet: add nodes; connections are memory-bound; use epoll-based servers (Go, Netty, Elixir/Phoenix) → 100k-1M per node.
Large groups: fan-out on write to 500 members is fine; for channels with 100k+ members switch to fan-out on read (clients pull from the conversation partition) plus a lightweight "new message" notify.
Hot partitions in Cassandra: month bucketing; avoid a single celebrity channel partition.
Presence storms: heartbeat every 30 s × 10M = 333k writes/s to Redis → batch, use TTL, and only publish status changes to online friends; lazy presence for large friend lists.
Push notifications: Kafka → notification workers → APNs/FCM with rate limits and collapse keys.
Media: never through the chat path; presigned S3 upload, CDN download, thumbnails async.
Search/history: optional Elasticsearch fed by Kafka (not possible with E2EE server-side).

8. Words to say#

WebSocket gateway, L4 load balancer, session registry, user→node mapping, Redis Pub/Sub fan-out, store-and-forward, Snowflake message_id, per-conversation ordering, clientMsgId dedupe, at-least-once + idempotent client, Cassandra partition key (conversation_id) + clustering (message_id DESC), month bucketing, wide partition, heartbeat / ping-pong, presence TTL, read receipts, multi-device cursor sync, APNs/FCM, fan-out on write vs read for huge channels, presigned URL + CDN, E2EE trade-off (no server-side search).