A request can fail to receive a response even after its work has completed. Retry behavior needs a definition of completion that survives that uncertainty.
A timeout leaves an unanswered question
Consider a reservation request. The service commits the reservation, but the connection closes before the client receives the response. From the client’s perspective, the result is unknown. Sending the same request again should help it discover the outcome without creating another reservation.
I start by separating a delivery attempt from the operation it represents. An attempt is something the transport can repeat. An operation is the business intent whose effects must remain consistent.
Give the operation a stable identity
A useful operation key has a defined scope, such as the authenticated account and the type of action. The server also needs to detect reuse of that key with different inputs. Otherwise, two distinct requests can accidentally share an outcome.
A preliminary lookup is not sufficient protection against concurrency: two requests can both observe that a record is absent. Enforcement belongs at the authoritative write boundary. A database uniqueness constraint can enforce uniqueness for the operation key, while the surrounding transaction defines which state changes belong together. PostgreSQL’s constraint documentation describes the database mechanism.
Make recovery part of the contract
For an operation entirely inside one database, recording the business change and its completion outcome in the same transaction provides a useful recovery boundary. A retry can then recover the recorded result. Long-running work needs an explicit in-progress state, an owner, and a recovery rule; a permanently stuck status is not a recovery strategy.
When a committed change must also produce a message, an outbox can store the message alongside the change and publish it later. Delivery can still repeat, so the consumer needs its own duplicate-handling boundary. An outbox does not make an arbitrary external side effect exactly-once. AWS explains this pattern and its delivery considerations.
The failure cases worth reviewing
- Two attempts arrive before either one completes.
- The operation commits, but the response never reaches the caller.
- The same key arrives with different inputs or under a different account.
- A retry arrives after its operation record has expired.
- An external side effect succeeds, but its acknowledgement is lost.
Retention is a business decision as well as a storage decision. If the system forgets an operation before callers stop retrying it, its duplicate protection has a smaller lifetime than the operation it protects.
The design question I want answered is precise: after any interruption, where can we find the authoritative outcome, and who is allowed to advance it?