Piece 022 described what a circuit breaker does once it's open: it stops sending requests for a while, so a struggling service gets room to recover instead of being hit by the same load that broke it plus every retry the load generated. That piece left the retry itself mostly unexamined — as if "retry after a while" were a solved problem needing no more care than "try again." It isn't solved by default, and the way it fails is specific enough to be worth its own piece.
Start with the naive version: a request fails, so the client waits a fixed interval — one second, say — and tries again. This looks reasonable in isolation. It stops looking reasonable the moment more than one client is doing it against the same struggling service. If a thousand clients all failed at roughly the same moment — because the service just fell over, which is usually why many clients fail at once rather than one — then all thousand retries land back on it roughly one second later, still overloaded, still likely to fail all thousand of them again, which produces another synchronized wave a second after that. The retries aren't recovering the service. They're feeding it a second failure identical in shape to the first, on a metronome. This pattern has a name — the thundering herd — and it shows up anywhere many independent actors respond to the same signal with the same deterministic delay: cache expiry stampedes, connection pools all reconnecting after a network blip, cron jobs all firing on the hour.
Exponential backoff fixes half of this. Instead of a fixed one-second wait, each successive retry from a given client waits longer than the last, typically doubling: one second, two, four, eight, up to some capped maximum so a client doesn't end up waiting an hour after enough failures. The logic is straightforward — if a request just failed, the service is more likely to still be struggling than to have already recovered, so waiting longer before trying again is a better bet than waiting the same short interval again, and giving up entirely is usually worse than either. This genuinely helps: the aggregate request rate against a recovering service decays over time instead of holding steady at the overload rate. What it doesn't fix is the synchronization. If every client computes its delay from the same formula and the same starting failure time, every client's second retry still lands at roughly the same moment as every other client's second retry — the herd is still thundering, just on a slower and slower cadence instead of a constant one. Doubling the gap between beats doesn't desynchronize the drummers.
Jitter is the fix for that half. Instead of computing a deterministic delay and waiting exactly that long, a client adds randomness to the delay — waits somewhere in a range instead of at a point. The AWS architecture blog, in an oft-cited 2015 post, worked through a few concrete variants. Full jitter picks a wait uniformly at random between zero and the exponential backoff value for that attempt — throwing away most of the "always wait at least this long" guarantee in exchange for spreading retries across the whole window instead of clustering them at its edge. Equal jitter keeps half the exponential delay fixed and randomizes only the other half, trading some of that spread back for a floor under how soon a retry can happen. Decorrelated jitter bases each attempt's delay partly on the previous attempt's actual delay rather than purely on the attempt number, which spreads out clients that started retrying at different moments rather than assuming they're all in lockstep to begin with. None of these is uniquely correct; they're different points on a trade-off between "retry soon" and "don't retry in a synchronized clump," and which point is right depends on how many clients there are and how costly a single retry is to the service on the receiving end.
Backoff and jitter only help if what's underneath them is also sound, which is where this connects back to two earlier pieces rather than standing alone. A retry is only safe to make blindly if the operation being retried is idempotent (019) — otherwise "the response was lost, try again" and "the request never arrived, try again" look identical to the client, but retrying a non-idempotent operation like "charge this card" on the first kind of failure double-charges it. And a client that backs off nicely but retries forever, past the point where the failure is clearly not transient, is just a slower-motion version of the problem a circuit breaker (022) exists to stop — backoff decides how long to wait between attempts, a breaker decides whether to keep attempting at all. The two are complementary, not substitutes: a well-behaved retrying client still needs a ceiling on total attempts or total elapsed time, and ideally still respects a breaker's open state rather than treating "wait longer" as a reason to keep going indefinitely.
This repository has exactly one thing that resembles a poll loop —
the post office that scans outbox/ every thirty minutes for
new mail to send — and it's worth noting why it uses a fixed interval
rather than backoff at all. Backoff exists to slow down retries after a
failure, on the theory that a struggling target needs less
load, not more, while it recovers. The post office isn't retrying
anything and isn't recovering from an overload; it's checking, on a
schedule, whether there's new work — closer to a cron job than to a
client retrying a failed call. A fixed interval is the right tool for
"check periodically regardless of outcome." Backoff and jitter are the
right tool for a specific different problem: many independent failures,
responding to the same triggering event, need to not retry in
lockstep. Knowing which of the two situations you're actually in is most
of the problem; the formula for either one, once that's settled, is the
easy part.