Retry and Timeout Patterns
Setting timeouts and deadlines, retrying safely with exponential backoff and jitter, retry budgets, idempotency and avoiding retry storms across layers.
Timeouts
Every remote call needs a timeout. Without one, a hung dependency holds resources indefinitely.
- Connect timeout (short, e.g. 100–500 ms) and read/response timeout (based on dependency p99.9 plus margin).
- Deadline propagation: pass the remaining time budget downstream (gRPC deadlines, custom headers) so lower layers do not keep working on requests the caller has abandoned.
- Caller timeout must be greater than the sum of downstream timeouts and retries on its critical path, or work is wasted.
Retries
Retry only when:
- The failure is transient (timeout, 503, connection reset, throttling 429 with Retry-After).
- The operation is idempotent or protected by an idempotency key.
- There is time budget left.
Use exponential backoff with full jitter: sleep = random(0, min(cap, base × 2^attempt)) (described in the AWS Architecture Blog post on backoff and jitter).
flowchart LR
A[Call] --> B{Success?}
B -- yes --> Done
B -- no --> C{Retryable error<br/>and idempotent?}
C -- no --> Fail[Fail fast]
C -- yes --> D{Attempts left<br/>and budget left?}
D -- no --> Fail
D -- yes --> E[Wait: random backoff<br/>with jitter] --> AJava example (Spring RestClient + Resilience4j Retry)
RetryConfig retryConfig = RetryConfig.custom()
.maxAttempts(3)
.intervalFunction(IntervalFunction.ofExponentialRandomBackoff(Duration.ofMillis(100), 2.0, 0.5, Duration.ofSeconds(2)))
.retryOnException(e -> e instanceof ResourceAccessException
|| (e instanceof HttpServerErrorException h && h.getStatusCode().value() == 503))
.build();
Retry retry = Retry.of("inventory", retryConfig);
var factory = new JdkClientHttpRequestFactory(HttpClient.newBuilder().connectTimeout(Duration.ofMillis(300)).build());
factory.setReadTimeout(Duration.ofMillis(800));
RestClient client = RestClient.builder().requestFactory(factory).baseUrl("http://inventory").build();
Stock stock = Retry.decorateSupplier(retry,
() -> client.get().uri("/stock/{sku}", sku).retrieve().body(Stock.class)).get();Retry storms and budgets
If each of three layers retries three times, the bottom service receives up to 27 attempts per user request during an outage. Mitigations: retry at one layer only, retry budgets (e.g. retries may add at most 10% extra load, a technique used in Finagle and Envoy), and circuit breakers to stop retrying a dead dependency.
Pros and cons
| Pros | Cons |
|---|---|
| Masks transient failures | Amplifies load during outages if uncontrolled |
| Simple to add | Duplicate side effects without idempotency |
| Timeouts bound resource usage | Wrong timeouts cause false failures or slow failure |
Interview questions
Should you retry a POST that creates a payment?
Only with an idempotency key that the server honours, so a retry returns the original result instead of creating a second payment. Without it, query status first instead of retrying.
How do you choose a timeout value?
From the dependency's measured latency distribution (e.g. slightly above p99.9) and the caller's total latency budget, then validate in load tests. Revisit when dependencies change.