Nobody has started this yet — be first.
Business impact
This is the subtlest bug in the set: it doesn't reproduce every time, which is exactly what makes it dangerous in production. A leader that steps down (graceful shutdown, deploy, rolling restart) will usually release the lock cleanly -- but on an unlucky timing coincidence between its own poll ticker and the shutdown signal, it can leak the lock instead, leaving the cluster with no cron-materialization leader until the leaked connection eventually times out or the process restarts again. Because it's intermittent, this is the kind of bug that passes staging and most production deploys, and then one day, during exactly the kind of high-traffic rolling restart where reliable failover matters most, no instance ever picks up leadership and every cron job silently stops materializing new runs.
Problem
pkg/leader/leader.go's Elector.Run loop selects between ctx.Done() and ticker.C. attempt uses the loop's own ctx (the one that gets cancelled on shutdown) for its connection health check: e.conn.Ping(ctx). When ctx is cancelled at almost exactly the moment the ticker also fires, Go's select has no defined preference between two simultaneously-ready cases -- it can pick the ticker.C branch instead of returning, running attempt one more time with an already-cancelled (or about-to-be-cancelled) ctx. Ping(ctx) then fails because ctx is done, not because the connection actually died -- but attempt can't tell the two apart, so it takes the "the connection is presumably dead, don't bother unlocking" branch. The still perfectly-healthy connection goes back to the pool with the advisory lock silently still held on it.
Current behavior
In a repeated-trials test (acquire-then-cancel cycled many times with a fast ticker), some rounds end with the advisory lock still held after the instance stepped down -- a brand-new, independent connection cannot acquire pg_try_advisory_lock.
Expected behavior
The connection health check inside attempt should be independent of the caller's shutdown signal, so a shutdown-in-progress is never misdiagnosed as a dead connection.
Steps to reproduce
This is a genuine race, not a one-shot reproduction -- the dedicated test uses a very fast tick (2ms) and repeats the acquire-then-cancel cycle 40 times to make hitting the race close to certain.
Why this matters
This is a real example of conflating two different meanings of "context cancelled": "the operation should stop because the caller gave up" versus "the operation failed because something is actually broken." Ping can't distinguish them from the returned error alone -- the caller has to pass a context that only carries the second meaning.
Suggested approach
attempt already creates a short-lived, independent context (qCtx) for exactly this reason, and uses it for the pool.Acquire and pg_try_advisory_lock calls further down. Check which context the Ping call a few lines above those is using.
Acceptance criteria
Verification
TEST_DATABASE_URL="postgresql://chronos:chronos@localhost:5432/chronos?sslmode=disable" go test -race ./tickets/tests/... -run TestTicket15 -v
Hints (0/2)
Try it without hints first — the reading is the exercise.
Working on this ticket
Work on a branch named for the ticket — that's what you'll submit.
Branch off your fork
Sign in to create it automatically, or do it yourself:
$git checkout -b fix/leader-ping-ctx-raceFix it and commit
Meet every acceptance criterion, and add a test that would have caught this.
Push the branch
$git push -u origin fix/leader-ping-ctx-raceSubmit it below
Paste your fork URL and the branch name, with a short write-up of the root cause.
Questions
Ask about anything unclear in the ticket — the maintainer and anyone who has solved it can answer. Please don't post full solutions.
Sign in to ask a question or reply.
Sign inhttp://localhost:8081app-1http://localhost:8082app-2http://localhost:8090curl localhost:8081/healthzcurl localhost:8081/readyzWork the tickets in tickets/docs/ (ordered easy → hard); each names one Go test in tickets/tests/. Most tickets need a real Postgres (tickets 4, 11, and 18 are pure-function tests and do not):
export TEST_DATABASE_URL="postgresql://chronos:chronos@localhost:5432/chronos?sslmode=disable"
go test ./tickets/tests/... -run TestTicket08 -v # a single ticket
./tickets/run_tickets.sh # all 20, clean pass/fail summary
Use -race when working on tickets 15, 16, or 17 (the concurrency ones) -- run_tickets.sh already does this for every ticket. Tickets 15 and 16 are adjacent bugs in the same function (pkg/leader/leader.go's lock-release path); fix ticket 16 first, since ticket 15's test keeps failing for ticket 16's reason until that one is fixed too.
Level 1
Fix a bug
Read existing behaviour, correct it.