Services

Resources

Company

Circuit breaker

A circuit breaker is a resilience pattern that wraps calls to a remote dependency and stops sending them once failures cross a threshold. While it is open, calls fail immediately or return a fallback instead of waiting on timeouts. After a cool-down it lets a few trial calls through to test whether the dependency has recovered.

How it works

The breaker has three states:

  • Closed: calls pass through and failures are counted.

  • Open: calls fail fast. For example, it opens when 50% of the last 20 calls fail and stays open for 30 seconds.

  • Half-open: a few trial calls go through. Success closes the breaker and failure opens it again.

Without a breaker, a service taking 200 requests per second to a dependency with a 10 second timeout ends up with about 2,000 requests waiting at once, by Little's law.

Why it matters in production

Breakers stop one slow dependency from exhausting threads, connections and memory in every caller, which is how cascading failures start. They also give the failing service room to recover instead of a retry storm. Expose breaker state as a metric so on-call engineers can see when one is open.

See also

On the One2N blog

Further reading

Comic about circuit breakers: a slow payments service leaves 2,000 calls waiting and search fails too; the breaker moves closed, open, half-open with trial calls; open, checkout returns a fallback in 5 ms and the page stays up.