Circuit Breaker

Failing fast when a dependency is unhealthy, the closed/open/half-open state machine, tuning thresholds, fallbacks and Resilience4j configuration.

Intent

Popularized by Michael Nygard's Release It!, a circuit breaker wraps calls to a remote dependency. When failures exceed a threshold, it opens and fails calls immediately for a cool-down period, protecting both the caller (threads, latency) and the struggling dependency (load). After the cool-down it half-opens to test recovery.

stateDiagram-v2
  [*] --> Closed
  Closed --> Open: failure rate or slow-call rate above threshold
  Open --> HalfOpen: wait duration elapsed
  HalfOpen --> Closed: trial calls succeed
  HalfOpen --> Open: trial calls fail

Java example (Resilience4j, programmatic)

CircuitBreakerConfig config = CircuitBreakerConfig.custom()
    .slidingWindowType(SlidingWindowType.COUNT_BASED)
    .slidingWindowSize(50)
    .minimumNumberOfCalls(20)
    .failureRateThreshold(50)                 // % of failed calls
    .slowCallDurationThreshold(Duration.ofMillis(500))
    .slowCallRateThreshold(60)                // % of slow calls also opens the breaker
    .waitDurationInOpenState(Duration.ofSeconds(10))
    .permittedNumberOfCallsInHalfOpenState(5)
    .recordExceptions(IOException.class, TimeoutException.class)
    .ignoreExceptions(BusinessValidationException.class)   // 4xx-like errors are not dependency failures
    .build();

CircuitBreaker cb = CircuitBreaker.of("recommendations", config);
Supplier<List<Product>> guarded = CircuitBreaker.decorateSupplier(cb, () -> recoClient.top(userId));
List<Product> result = Try.ofSupplier(guarded).getOrElse(List.of());   // fallback: empty carousel

Tuning guidance

  • Count only dependency failures (timeouts, 5xx, connection errors), not client errors.
  • Include slow calls: a dependency answering in 10 s is as harmful as one that fails.
  • Minimum call volume avoids opening on a handful of requests at night.
  • Expose breaker state as metrics and alert on prolonged open state.

Pros and cons

Pros Cons
Prevents cascading failures and thread exhaustion Wrong thresholds cause flapping or late reaction
Gives dependencies room to recover Fallbacks may hide outages if unmonitored
Fast failure improves user-perceived latency Per-instance breakers see only local traffic

Interview questions

Where should circuit breakers live: library or service mesh?

Mesh-level outlier detection and ejection protect at the connection level without code changes; library breakers know business context and can choose meaningful fallbacks. Many systems use both, with the application owning fallback behaviour.

What makes a good fallback?

Cached or stale data, a default value, a reduced feature (hide the widget), or a queued request for later processing. Never a fallback that silently produces wrong financial results.