System Design: Real-Time Food Delivery Tracking

Requirements

Functional Requirements:

• Display live driver location on a map during active deliveries
• Show real-time order status transitions (confirmed, preparing, picked up, en route, delivered)
• Recalculate and display updated ETA as delivery progresses
• Push notifications at key milestones (driver assigned, food picked up, arriving)
• Support delivery photo proof upon completion
• Historical delivery tracking for past orders

Non-Functional Requirements:

• Support 2M concurrent active deliveries with live tracking
• Location update latency from driver to consumer under 3 seconds
• ETA recalculation within 5 seconds of significant route deviation
• 99.95% availability for the tracking service
• Graceful degradation when location services are unavailable

Scale Estimation

2M concurrent deliveries with driver location updates every 4 seconds = 500K location messages/sec inbound. Each delivery has 1 consumer tracking it on average, so 500K location updates/sec need to be pushed outbound to consumers. Add order status events (15 per order lifecycle × 2M active / 45-min avg delivery = 11K status events/sec). Total outbound message throughput: ~511K messages/sec via WebSocket. Each location payload is roughly 200 bytes (lat, lng, heading, speed, timestamp) = 100MB/sec inbound data. WebSocket connections: 2M drivers + 2M consumers = 4M concurrent connections.

High-Level Architecture

The tracking system is a real-time data pipeline with three layers: ingestion, processing, and delivery. The ingestion layer receives driver location updates via a lightweight UDP-based protocol (falls back to HTTPS POST on networks that block UDP) at edge servers geographically distributed to minimize latency. Edge servers batch location updates into Kafka with a 500ms flush interval, partitioned by driver_id for ordering guarantees.

The processing layer consists of Flink stream processing jobs that consume from Kafka and perform three tasks: (1) geofence detection — determining when a driver enters the restaurant pickup zone or the delivery drop-off zone to trigger automatic status transitions; (2) ETA recalculation — comparing the driver's current position and trajectory against the planned route and updating the predicted arrival time; (3) anomaly detection — flagging deliveries where the driver has been stationary for more than 5 minutes or is moving away from the destination.

The delivery layer maintains WebSocket connections to all active consumers via a horizontally scaled WebSocket Gateway fleet. Each gateway server holds ~50K connections. When a processed location update arrives (via internal Kafka topic tracking-updates), a Router Service determines which gateway server holds the consumer's connection and forwards the update. Consumers who have backgrounded the app receive condensed updates via APNS/FCM push notifications at key milestones only.

Core Components

Location Ingestion Service

Driver apps send GPS coordinates every 4 seconds using a custom binary protocol over UDP for minimal overhead and battery efficiency. The payload includes driver_id, latitude, longitude, heading, speed, accuracy, and timestamp. Edge servers in 40+ PoPs worldwide receive these packets, validate the driver_id against a local authentication cache, apply basic noise filtering (reject updates with accuracy >100m or speed >200km/h), and batch them into Kafka. A separate HTTPS fallback endpoint handles environments where UDP is blocked. The service processes 500K updates/sec with p99 latency under 50ms at the edge.

WebSocket Gateway

The WebSocket Gateway is a stateful service built on Netty (Java) optimized for high connection density. Each server maintains 50K concurrent WebSocket connections using epoll for efficient I/O multiplexing. Connection state (which delivery_id each consumer is tracking) is stored in a local ConcurrentHashMap and replicated to Redis for failover. When a gateway server needs to drain for deployment, it sends a REDIRECT frame to connected clients with the address of a healthy server, enabling zero-downtime deployments. The gateway fleet sits behind an L4 load balancer (AWS NLB) with sticky sessions based on a connection token.

ETA Calculation Engine

ETA is recalculated every time a new driver location arrives. The engine maintains a route plan for each active delivery (fetched from a routing service like OSRM or Google Directions API at dispatch time). As location updates stream in, the engine snaps the driver's position to the nearest road segment on the planned route and computes remaining distance. The base ETA uses historical speed data for each road segment by time-of-day (stored in a Redis-backed lookup table partitioned by H3 hex cells). A correction factor from real-time traffic conditions (sourced from aggregated driver speed data across the fleet) adjusts the estimate. The final ETA is smoothed using exponential moving average to prevent jittery display updates.

Database Design

Active delivery tracking state is stored entirely in Redis for performance. Each active delivery has a Redis hash tracking:{delivery_id} containing driver_lat, driver_lng, driver_heading, current_status, eta_seconds, route_polyline, and last_updated. This hash is written by the Flink processing layer and read by the WebSocket Gateway for push updates. TTL is set to 2 hours (max expected delivery time + buffer). Historical location breadcrumbs are written to a time-series database (TimescaleDB) partitioned by day, with columns: delivery_id, driver_id, lat, lng, heading, speed, timestamp. This data is retained for 90 days for dispute resolution and ETA model training.

Order status transitions are stored in the Order Service's PostgreSQL database as an append-only events table: delivery_id, status, timestamp, metadata (JSONB). This event log is the source of truth; the current status is a materialized view over the latest event per delivery.

API Design

• GET /api/v1/tracking/{delivery_id}/connect — Upgrade to WebSocket; server streams location and status updates as JSON frames
• POST /api/v1/tracking/locations — Driver submits batch of location updates; body is an array of {lat, lng, heading, speed, timestamp} objects
• GET /api/v1/tracking/{delivery_id}/status — REST fallback for current delivery status, driver location, and ETA (for clients that cannot use WebSocket)
• POST /api/v1/tracking/{delivery_id}/proof — Driver uploads delivery photo proof; multipart form with image and GPS coordinates

Scaling & Bottlenecks

The WebSocket Gateway is the most operationally complex component. With 4M concurrent connections, the fleet requires ~80 servers (50K connections each). The challenge is connection redistribution during scaling events — adding a new server means no connections are routed to it until existing ones disconnect and reconnect. A proactive rebalancing mechanism periodically disconnects 1% of connections from overloaded servers, causing clients to reconnect via the load balancer to a less loaded server.

Kafka throughput for location data (500K messages/sec) requires a dedicated cluster with 30+ partitions per topic, SSD-backed brokers, and consumer groups sized to match partition count. The Flink ETA calculation job is the most CPU-intensive — snapping GPS coordinates to road segments requires a spatial index (R-tree) of the road network loaded into each Flink task manager's memory (~2GB per city). Horizontal scaling is achieved by partitioning the Flink job by city/region, each running independently.

Key Trade-offs

• UDP over HTTPS for location ingestion: UDP eliminates TCP handshake overhead and head-of-line blocking, reducing battery drain on driver phones by 20% — the trade-off is potential packet loss (mitigated by 4-second frequency; missing one update is acceptable)
• Redis for active tracking state over a database: Sub-millisecond reads from Redis enable real-time push updates, but Redis is not durable — mitigated by treating the Order Service's PostgreSQL event log as the source of truth, with Redis as a derived cache
• Pre-computed route-based ETA over live routing API calls: Fetching routes once at dispatch and snapping to them avoids hammering the routing API at 500K/sec, but stale routes (due to road closures or traffic) degrade accuracy — a route refresh is triggered when the driver deviates more than 500m from the planned path
• Exponential smoothing on ETA display: Prevents jittery ETA changes that frustrate users (jumping between 12 min and 8 min repeatedly), but introduces a lag in reflecting genuine ETA changes — tuned with a smoothing factor of 0.3