Circuit Breaker Pattern

The circuit breaker pattern prevents an application from repeatedly calling a failing downstream service, giving the service time to recover while returning fast failures to callers — stopping cascading failures across a distributed system.

What It Really Means

In a microservices architecture, Service A calls Service B, which calls Service C. If Service C becomes slow or unresponsive, requests back up in Service B. Service B's thread pool fills up. Now Service B cannot serve requests either. Service A's requests to B start timing out. The failure cascades through the entire system from a single point of failure.

The circuit breaker pattern, popularized by Michael Nygard in "Release It!" (2007), works exactly like an electrical circuit breaker. When a downstream service fails too many times, the circuit "opens" and all subsequent calls fail immediately without even attempting the request. After a cooldown period, the circuit enters a "half-open" state and allows a few test requests through. If those succeed, the circuit closes and normal operation resumes. If they fail, the circuit opens again.

This pattern is implemented in Netflix Hystrix (now in maintenance mode), Resilience4j (the modern Java alternative), Polly (.NET), and is built into service meshes like Istio and Envoy. If you are building microservices, you will use circuit breakers.

How It Works in Practice

The Three States

Closed (normal operation): Requests pass through to the downstream service. The circuit breaker tracks the error rate. If the error rate exceeds a threshold (e.g., 50% of the last 100 requests), the circuit transitions to Open.

Open (failing fast): All requests are immediately rejected without calling the downstream service. The caller receives a fallback response or an error. After a configurable timeout (e.g., 30 seconds), the circuit transitions to Half-Open.

Half-Open (testing recovery): A limited number of requests (e.g., 5) are allowed through to the downstream service. If they succeed, the circuit closes. If any fail, the circuit opens again.

Real-World: Netflix Hystrix

Netflix built Hystrix because a single slow dependency could bring down their entire streaming platform. With hundreds of microservices, each calling multiple downstream services, cascading failures were existential.

Hystrix wrapped every inter-service call in a circuit breaker. When the recommendation service became slow, Hystrix opened the circuit and returned cached recommendations or a generic "trending" list. The streaming service continued working — users saw slightly degraded recommendations instead of a blank screen.

Hystrix also introduced the bulkhead pattern: each downstream dependency gets its own thread pool. If the payment service is slow, it fills its own 20-thread pool but does not affect the 20-thread pool allocated for the inventory service.

Real-World: Envoy Service Mesh

Envoy proxy implements circuit breaking at the infrastructure layer. Instead of adding circuit breaker code to every service, you configure Envoy to monitor error rates and open circuits automatically. This is particularly powerful in Kubernetes environments where Envoy runs as a sidecar proxy for every pod.

Envoy's circuit breaker tracks: max connections, max pending requests, max requests, and max retries — opening the circuit when any threshold is exceeded.

Implementation

Trade-offs

Advantages

• Prevents cascading failures: A single slow service does not bring down the entire system
• Fast failure: Callers get immediate responses instead of waiting for timeouts
• Self-healing: The half-open state automatically tests recovery without manual intervention
• Resource preservation: Prevents thread pool exhaustion and connection pool depletion

Disadvantages

• False positives: Transient errors can open the circuit unnecessarily, blocking healthy requests
• Tuning complexity: Threshold, window size, and recovery timeout require careful tuning per dependency — wrong values either trigger too eagerly or too late
• Testing difficulty: Circuit breaker behavior is hard to test in integration environments because you need to simulate partial failures
• Fallback design: Every circuit-broken call needs a meaningful fallback, which is not always obvious (what is the fallback for a payment service?)

Common Misconceptions

• "Circuit breakers replace retries" — They complement each other. Retries handle transient errors (one bad request). Circuit breakers handle sustained failures (the service is down). Use retries within a circuit breaker, not instead of one.

• "One circuit breaker per service is enough" — You need one circuit breaker per dependency endpoint. If Service A calls Service B's /users and /orders endpoints, each needs its own circuit breaker because they may have different failure modes.

• "The circuit breaker fixes the downstream service" — It does not. The circuit breaker protects the caller and gives the downstream service breathing room to recover. Someone still needs to fix the actual problem.

• "Circuit breakers are only for HTTP calls" — They apply to any remote dependency: database connections, message queues, gRPC calls, third-party APIs, file system operations on network-mounted storage.

• "Service meshes make application-level circuit breakers unnecessary" — Service mesh circuit breakers (Envoy, Istio) operate at the connection/request level. Application-level circuit breakers can implement domain-specific fallback logic that infrastructure cannot.

How This Appears in Interviews

Circuit breakers are a standard topic in microservices and system design interviews:

• "How do you prevent cascading failures?" — Describe the circuit breaker pattern with its three states, then mention bulkheads, timeouts, and retry budgets as complementary patterns. See our system design interview guide.
• "Design a resilient payment processing system" — use circuit breakers around the payment gateway, with fallbacks that queue payments for later processing.
• "Your service is intermittently failing. How do you diagnose whether it is the circuit breaker?" — check circuit breaker metrics (open/close events, error rates), examine whether the circuit breaker threshold is too aggressive, and verify the recovery timeout.
• "Compare circuit breaker and retry" — retries for transient errors, circuit breakers for sustained failures, and explain how they work together.

Related Concepts

• Saga Pattern — Handling distributed transactions with compensating actions when circuits break
• Eventual Consistency — Fallback responses during open circuits may serve stale data
• CAP Theorem — Circuit breakers implicitly trade consistency for availability
• Gossip Protocol — How service meshes detect failing nodes
• System Design Interview Guide — Framework for resilience discussions
• Algoroq Pricing — Practice microservices design interview questions