Design a circuit breaker for a service that calls a downstream HTTP dependency. The breaker must track failure rate, trip when the threshold is breached, fail fast while open, and recover via half-open probe calls.
What to decide
-
Failure detection window — How far back do you look? A sliding window (last 10 seconds) adapts faster than a fixed window but uses more memory. A 1-minute window smooths out bursts but delays trip detection.
-
Failure threshold — 50% of requests failing within the window is common, but the right number depends on the service's normal error rate. A payment service with a 0.1% baseline error rate should trip at 5%; a search endpoint with a 5% baseline should trip at 50%.
-
Cooldown period — How long to wait in the open state before probing? Too short and you hammer a recovering dependency with retries. Too long and you deny service unnecessarily. A randomized cooldown (30-90 seconds) avoids synchronized recovery storms.
-
Half-open probe — Send exactly one request through the breaker. If it succeeds, close the circuit. If it fails, reopen. Multiple concurrent probes from multiple threads need a lock or semaphore to avoid double-open.
Implementation sketch
class CircuitBreaker:
def __init__(self, failure_threshold, cooldown, window):
self.state = "closed" # closed | open | half_open
self.failures = deque() # (timestamp, outcome) within window
self.threshold = failure_threshold
self.cooldown = cooldown
self.window = window
def call(self, fn, *args, **kwargs):
if self.state == "open":
if time.time() - self.opened_at > self.cooldown:
self.state = "half_open"
return self._attempt(fn, *args, **kwargs)
raise CircuitOpenException()
try:
result = fn(*args, **kwargs)
self._record_success()
return result
except Exception as e:
self._record_failure()
raise
def _attempt(self, fn, *args, **kwargs):
try:
result = fn(*args, **kwargs)
self.state = "closed"
self.failures.clear()
return result
except Exception as e:
self.state = "open"
self.opened_at = time.time()
raise
def _record_failure(self):
now = time.time()
self.failures.append((now, True))
self.failures = [f for f in self.failures if f[0] > now - self.window]
if len(self.failures) / max(1, len(self.failures)) > self.threshold:
self.state = "open"
self.opened_at = now
def _record_success(self):
now = time.time()
self.failures.append((now, False))
self.failures = [f for f in self.failures if f[0] > now - self.window]Read the circuit breaker article for the design model, and the cross-region failover trace to see how circuit breakers drive failover decisions in production.