System Design: Slack

Requirements

Functional Requirements:

• Users send messages in channels (public/private) and direct messages
• Real-time message delivery with typing indicators and presence
• Full-text search across all message history within a workspace
• File sharing with preview generation (images, PDFs, code snippets)
• Threaded conversations within channels
• Rich integrations ecosystem (bots, webhooks, slash commands)

Non-Functional Requirements:

• 20 million DAU across 750,000 organizations
• Message delivery latency under 150ms for real-time feel
• 99.99% availability with zero message loss
• Search latency under 500ms for queries spanning years of message history
• Strict workspace-level data isolation for enterprise compliance

Scale Estimation

With 20M DAU sending an average of 50 messages each, Slack processes roughly 1 billion messages per day — approximately 11,500 messages per second. Each message averages 200 bytes of text, producing ~200GB of new text data daily. File uploads average 2MB each, with approximately 50M files uploaded daily, adding 100TB of storage per day. Real-time connections: 10M concurrent WebSocket connections during peak hours. Search index size grows by approximately 200GB/day; the total search corpus exceeds 500TB for long-running workspaces.

High-Level Architecture

Slack uses a multi-tenant architecture where each workspace is logically isolated but shares physical infrastructure. The real-time layer is built on WebSocket connections: each client connects to a Gateway Service that maintains the WebSocket, handles authentication via workspace-scoped OAuth tokens, and subscribes the connection to relevant channels using a pub/sub system. When a user sends a message, it flows through: Client → Gateway (WebSocket) → Message Service → MySQL (write) + Kafka (event) → Channel Fan-out Service → Gateway (push to all subscribed clients).

The Channel Fan-out Service reads channel membership from a Channel Service (backed by MySQL), determines which connected clients should receive the message, and publishes to the appropriate Gateway nodes via an internal pub/sub layer (originally using a custom system called Flannel, now migrated to a Kafka Streams-based system). Messages are persisted before fan-out, ensuring durability even if real-time delivery fails — clients reconcile missed messages on reconnect using a last-seen message ID.

Search is powered by a dedicated Elasticsearch cluster. Every message written to MySQL also emits a Kafka event consumed by a Search Indexer that writes to Elasticsearch. The search index is partitioned by workspace_id, ensuring queries never span workspace boundaries. This also enables per-workspace data retention policies and compliance holds required by enterprise customers.

Core Components

WebSocket Gateway Service

The Gateway Service manages all real-time WebSocket connections. Each Gateway node handles up to 500K concurrent connections using an event-driven architecture (originally Java/Netty, portions now in Go). On connection, the Gateway subscribes to all channels the user belongs to via an in-memory subscription map. The Gateway also handles presence aggregation — it knows which users are connected and publishes presence changes to subscribed channels using debounced updates (batched every 5 seconds to avoid presence storms).

Message Service

The Message Service is the write path for all messages. It validates the message (permissions, rate limits, content length), generates a globally unique message ID (Snowflake-style: timestamp + workspace_shard + sequence), writes to MySQL, and emits a Kafka event. The service also handles message edits and deletions — edits create a new version linked to the original message_id; deletions set a tombstone flag. Thread replies are stored as regular messages with a thread_ts field linking to the parent message.

Search Service

Slack's search uses Elasticsearch with workspace-level index sharding. Each workspace gets its own Elasticsearch index (or a shared index with workspace_id routing for smaller workspaces). The indexer processes Kafka events and writes documents containing message text, sender, channel, timestamp, and file metadata. Search queries support Slack-specific filters: from:@user, in:#channel, has:link, before:date. A query rewriting layer translates Slack search syntax into Elasticsearch DSL. Results are ranked by relevance (BM25) with a recency boost.

Database Design

Message data lives in a sharded MySQL cluster, partitioned by workspace_id. The Messages table schema: message_id (BIGINT, Snowflake), workspace_id, channel_id, user_id, text (TEXT), thread_ts (BIGINT nullable), edited_at, deleted (BOOL), attachments (JSON), created_at. A composite index on (channel_id, created_at) enables efficient paginated channel history loading. Channel membership is stored in a separate table: (channel_id, user_id, joined_at, notification_pref) with an index on (user_id, workspace_id) for listing a user's channels.

File metadata is stored in MySQL (file_id, workspace_id, uploader_id, filename, size, mime_type, storage_url, thumbnail_url). Actual file bytes are stored in S3 with per-workspace bucket prefixes for isolation. Workspace configuration (name, domain, plan, retention policy, SSO settings) lives in a separate MySQL cluster that serves as the control plane, queried infrequently but requiring strong consistency.

API Design

• POST /api/chat.postMessage — Send a message: {channel, text, thread_ts?, attachments?}; returns message with ts identifier
• GET /api/conversations.history?channel={id}&cursor={ts}&limit=100 — Fetch channel history with cursor-based pagination
• GET /api/search.messages?query={q}&sort=relevance&count=20 — Full-text search within workspace
• POST /api/files.upload — Upload file with multipart form; triggers async thumbnail/preview generation

Scaling & Bottlenecks

The primary bottleneck is WebSocket connection density. With 10M concurrent connections, even at 500K per node, Slack needs 20+ Gateway nodes. Each connection consumes memory for the subscription set (average user belongs to 15 channels). Slack uses connection draining during deploys — new connections go to new nodes while existing connections are gradually migrated. A Presence Service aggregates online status across all Gateway nodes using a gossip protocol, publishing changes via Kafka.

MySQL sharding by workspace_id works well because most queries are workspace-scoped. However, very large workspaces (100K+ members with millions of messages) can create hot shards. Slack addresses this by sub-sharding large workspaces across multiple MySQL instances using channel_id ranges. Read replicas serve channel history reads, while writes always go to the primary. The Elasticsearch cluster scales independently, with dedicated hot-warm-cold architecture: recent messages on SSD-backed hot nodes, older messages on HDD-backed warm nodes.

Key Trade-offs

• MySQL over NoSQL for messages: Relational guarantees (strong consistency, transactions for edit/delete) are critical for enterprise compliance; the trade-off is more complex sharding compared to Cassandra
• Workspace-level search isolation: Per-workspace Elasticsearch indices ensure data isolation and enable per-workspace retention policies, but increase index management overhead compared to a global index
• WebSocket over long-polling: WebSockets provide true real-time delivery with lower latency than long-polling, but require persistent connection management and complicate horizontal scaling
• Kafka-based fan-out over direct pub/sub: Kafka provides durability and replay capability for message fan-out, enabling reliable delivery even during Gateway restarts, at the cost of slightly higher latency than in-memory pub/sub