System Design: Driver Dispatch System

Requirements

Functional Requirements:

• Dispatch system sends trip/delivery offers to drivers and tracks responses
• Drivers can accept, decline, or ignore offers within a configurable timeout
• System tracks driver state: offline, available, en-route to pickup, on-trip
• Automatic re-dispatch to next best driver on decline or timeout
• Dispatcher supports priority queues (premium rides dispatched first)
• Full audit log of all dispatch events for compliance and debugging

Non-Functional Requirements:

• Offer delivery latency under 200ms from dispatch trigger to driver app
• Re-dispatch completes within 1 second of decline/timeout
• Support 5 million concurrent driver connections globally
• System must be idempotent — duplicate events must not cause double dispatch
• 99.99% availability for dispatch pipeline; failures must not lose requests

Scale Estimation

For a platform with 5 million active drivers globally, each maintaining a persistent connection to the dispatch gateway, we need ~5 million concurrent WebSocket or long-poll connections. Connection state memory: ~10 KB per connection × 5 million = 50 GB — manageable across a fleet of ~500 gateway nodes at 100 MB each. Dispatch events: 10 million trips/day × average 1.5 dispatch attempts/trip = 15 million dispatch events/day = ~175/second average, ~5,000/second peak.

High-Level Architecture

The dispatch system consists of three layers: the Connection Gateway (maintaining persistent connections with driver apps), the Dispatch Orchestrator (the core state machine driving the offer workflow), and the Notification Delivery layer (actually pushing offers to driver devices).

Drivers connect to the Connection Gateway — a fleet of stateful WebSocket servers load-balanced by a consistent-hash NLB that pins each driver to a specific gateway node. The gateway node holds the connection and routes incoming messages (heartbeats, responses) to the appropriate backend service via a message bus.

When the Matching Engine selects a driver, it publishes a dispatch event to the Dispatch Orchestrator via Kafka. The Orchestrator creates a dispatch record in Cassandra, sets a TTL timer (8 seconds), and instructs the Notification Delivery layer to push the offer to the driver's connection. The Orchestrator then awaits either a response event or a timeout event to drive the FSM transition.

Core Components

Connection Gateway

Built on Netty (Java) or uWSGI (Python), each gateway node handles ~10,000 concurrent WebSocket connections. Driver connections are pinned to gateway nodes using consistent hashing on driver_id — this means the same driver always reconnects to the same node cluster, simplifying state lookup. A heartbeat protocol (client pings every 30 seconds) detects stale connections. The gateway maintains a Redis hash of (driver_id → gateway_node_id) so that the Orchestrator can route messages to the right gateway.

Dispatch Orchestrator

The Orchestrator is a Kafka Streams application that maintains per-request FSM state in a local RocksDB state store (backed up to Kafka changelog topics for fault tolerance). States: PENDING_MATCH → OFFER_SENT → ACCEPTED | DECLINED | TIMED_OUT. On DECLINED or TIMED_OUT, the Orchestrator re-publishes to the Matching Engine's re-dispatch topic with updated context (already-declined driver_ids excluded). On ACCEPTED, it emits a TRIP_CREATED event to downstream services.

Offer Push Service

Receives push instructions from the Orchestrator and delivers them via the appropriate channel: WebSocket (primary, sub-100ms latency), APNs/FCM (fallback for backgrounded apps, ~1-2 second latency), or SMS (last resort for feature phones). The service tracks delivery receipts: if the WebSocket push is not acknowledged within 500ms, it automatically falls back to push notification. All attempts are logged with timestamps for SLA monitoring.

Database Design

Dispatch records in Cassandra: partition key is (driver_id, date), clustering key is dispatch_timestamp DESC. Each record stores: request_id, trip_id, offer_sent_at, response_type (ACCEPTED/DECLINED/TIMEOUT), response_at, and gateway_node_id. A separate Cassandra table indexed by request_id supports the Orchestrator's idempotency check (has this request_id already been dispatched?). Driver state is maintained in Redis: (driver_id → {state, current_trip_id, location, last_seen}) with a 60-second TTL refreshed on each heartbeat.

API Design

• WebSocket /ws/driver/{driver_id} — Persistent connection for driver app; server pushes offer objects (JSON); client sends ACCEPT/DECLINE messages
• POST /v1/dispatch/offer — Internal API called by Matching Engine; accepts driver_id, request_id, trip details; triggers the dispatch FSM
• POST /v1/dispatch/response — Called by driver app HTTP fallback if WebSocket is unavailable; accepts driver_id, request_id, response_type
• GET /v1/dispatch/audit/{request_id} — Returns full dispatch timeline for a request; used by support and compliance teams

Scaling & Bottlenecks

The Connection Gateway is the scaling challenge: 5 million persistent WebSocket connections require careful memory management and connection pinning. Horizontal scaling adds more gateway nodes, with consistent hashing ensuring minimal connection redistribution on node addition. A connection registry in Redis maps driver_id to gateway node; this registry is the single point of coordination and is sharded across a Redis cluster.

The Orchestrator scales via Kafka partition count: each partition is owned by one Orchestrator instance with its own RocksDB state store. Adding partitions + Orchestrator instances is the scale-out path. Cross-partition state (e.g., checking if a driver is already on an active dispatch) is handled by a lookup table in Redis that all Orchestrator instances read.

Key Trade-offs

• Stateful gateway vs. stateless with external state — stateful WebSocket gateways (holding connection in memory) reduce latency but complicate failover; a secondary gateway can take over via the Redis connection registry
• Short vs. long offer timeout — 8 seconds is a balance: short enough to re-dispatch quickly, long enough for drivers in spotty network conditions to respond
• WebSocket vs. SSE vs. polling — WebSocket is bidirectional and lower latency, but HTTP/2 SSE is simpler and scales better for read-heavy notification workloads; Uber uses a proprietary framed protocol over HTTP/2
• At-least-once vs. exactly-once dispatch — Kafka at-least-once delivery with idempotency keys in Cassandra prevents double-dispatch without requiring expensive exactly-once semantics in the broker