Circuit breaker needs one more failure than configured to trip

Nobody has started this yet — be first.

Business impact

The circuit breaker exists so a dead upstream does not tie up gateway capacity waiting out repeated timeouts. A route configured with failure_threshold: 5 is supposed to trip after the 5th consecutive failure. Right now it takes a 6th -- meaning every route's breaker is quietly one failure more tolerant than whoever configured it intended. For a route tuned with a low threshold specifically to fail fast (failure_threshold: 1 or 2), that off-by-one is proportionally huge: a threshold: 1 route effectively behaves like threshold: 2, doubling how many failed requests get sent into an already-failing upstream before the breaker does its job.

Problem

Breaker.on_failure() trips the breaker with if self._consecutive_failures > self._failure_threshold. With failure_threshold=3, three consecutive failures leave _consecutive_failures == 3, and 3 > 3 is False -- the breaker stays closed. It takes a fourth failure to trip.

Current behavior

A route configured with circuit_breaker: {failure_threshold: 3} sends its 4th request to a failing upstream too, instead of short-circuiting it -- on_failure()'s > comparison against _failure_threshold is one failure too tolerant.

Expected behavior

With failure_threshold=3, the breaker trips on exactly the 3rd consecutive failure -- the request that pushes _consecutive_failures to 3 should be the last one that reaches the upstream before the next one gets short-circuited.

Steps to reproduce

Configure a route with circuit_breaker: {failure_threshold: 3} against an upstream that always fails.
Send 3 requests. All 3 reach the upstream and get its real error.
Send a 4th. Expected: 503, short-circuited, upstream never called. Actual: the 4th one also reaches the upstream, and it is the 4th failure that finally trips the breaker.

Why this matters

> vs >= on a counter comparison is one of the easiest single-character mistakes to make and one of the easiest to miss in review, because both versions "work" in the sense that the breaker does eventually trip -- just one failure later than configured, every time, for every route. Nothing about calling it with a fresh breaker and checking "did it end up open" catches this; you have to count exactly how many failures it took.

Suggested approach

Look at the comparison operator in on_failure() against what "threshold" should mean: is a failure_threshold of N the number of failures that are still tolerated, or the number of failures that trips it?

Acceptance criteria

With failure_threshold=N, the breaker opens on exactly the Nth consecutive failure, not the (N+1)th
failure_threshold=1 trips on the very first failure

Verification

pytest practice_tickets/tests/test_ticket_09_breaker_threshold_off_by_one.py -v

Hints (0/2)

Try it without hints first — the reading is the exercise.

Working on this ticket

Work on a branch named for the ticket — that's what you'll submit.

Branch off your fork

$git checkout -b fix/breaker-threshold-off-by-one

Fix it and commit

Meet every acceptance criterion, and add a test that would have caught this.

Push the branch

$git push -u origin fix/breaker-threshold-off-by-one

Submit it below

Paste your fork URL and the branch name, with a short write-up of the root cause.

Submit your fix

Questions

Stuck on something?

Ask about anything unclear in the ticket — the maintainer and anyone who has solved it can answer. Please don't post full solutions.

No questions yet. If something in this ticket reads ambiguously, you are probably not the only one — ask.

The project

Dynamic API Gateway & Reverse Proxy

A FastAPI reverse proxy that streams every request and response without ever buffering a body in full, with hot-reloadable routing, per-route auth, Redis token-bucket rate limiting, and a per-upstream circuit breaker.

Fork on GitHub

Download

Setup

Dynamic API Gateway & Reverse Proxy

git clone <your-fork>
cd FastAPIDevTraining/gateway_proxy
cp .env.example .env   # adjust GATEWAY_ADMIN_TOKEN etc. if you like
docker compose up --build   # Redis + the gateway (uvicorn --reload) + 2 demo upstreams

The gateway is reachable at . Two demo upstreams come up alongside it — (low-latency JSON echo at , plus for the streaming-passthrough demo) and (, , for timeout and circuit-breaker demos) — and wires five example routes against them, one per feature. Verify the stack with (liveness only) and (pings Redis, which every rate-limited route depends on). Every endpoint (route CRUD, reload, persist, breaker snapshot) requires .