Kafka vs Pulsar

Apache Kafka and Apache Pulsar are both distributed streaming platforms, but they differ fundamentally in architecture. Kafka couples storage and compute on broker nodes. Pulsar separates them — brokers handle serving, while Apache BookKeeper handles storage.

Architecture Deep Dive

Kafka's brokers own their partitions' data on local disk. When you add brokers, you must rebalance partitions, which involves copying data across the network. This coupling makes scaling straightforward at moderate scale but operationally expensive at very large deployments.

Pulsar's brokers are stateless — they serve topics but don't own data. BookKeeper stores data in segments called ledgers across bookie nodes. Adding brokers requires no data movement. Adding bookies automatically distributes new segments. This separation makes elastic scaling much simpler.

Multi-tenancy and Isolation

Pulsar was designed from the start for multi-tenancy. It supports tenants, namespaces, and per-namespace quotas, authentication, and authorization. You can run one Pulsar cluster for multiple teams or products.

Kafka achieves multi-tenancy through topic-level ACLs and quotas, but lacks native namespace isolation. Many organizations run separate Kafka clusters per team, increasing operational cost.

Performance Trade-offs

Kafka's co-located storage design means reads from the page cache are extremely fast for recent data. Pulsar's network hop to BookKeeper adds a small latency overhead, though tail latency is often better due to BookKeeper's quorum-based writes.

For stream processing and event-driven design, both platforms are capable. The choice often comes down to operational model preferences. Explore our system design interview guide for when each pattern matters.