Retry and Timeout Patterns

Setting timeouts and deadlines, retrying safely with exponential backoff and jitter, retry budgets, idempotency and avoiding retry storms across layers.

Timeouts

Every remote call needs a timeout. Without one, a hung dependency holds resources indefinitely.

  • Connect timeout (short, e.g. 100–500 ms) and read/response timeout (based on dependency p99.9 plus margin).
  • Deadline propagation: pass the remaining time budget downstream (gRPC deadlines, custom headers) so lower layers do not keep working on requests the caller has abandoned.
  • Caller timeout must be greater than the sum of downstream timeouts and retries on its critical path, or work is wasted.

Retries

Retry only when:

  1. The failure is transient (timeout, 503, connection reset, throttling 429 with Retry-After).
  2. The operation is idempotent or protected by an idempotency key.
  3. There is time budget left.

Use exponential backoff with full jitter: sleep = random(0, min(cap, base × 2^attempt)) (described in the AWS Architecture Blog post on backoff and jitter).

flowchart LR
  A[Call] --> B{Success?}
  B -- yes --> Done
  B -- no --> C{Retryable error<br/>and idempotent?}
  C -- no --> Fail[Fail fast]
  C -- yes --> D{Attempts left<br/>and budget left?}
  D -- no --> Fail
  D -- yes --> E[Wait: random backoff<br/>with jitter] --> A

Java example (Spring RestClient + Resilience4j Retry)

RetryConfig retryConfig = RetryConfig.custom()
    .maxAttempts(3)
    .intervalFunction(IntervalFunction.ofExponentialRandomBackoff(Duration.ofMillis(100), 2.0, 0.5, Duration.ofSeconds(2)))
    .retryOnException(e -> e instanceof ResourceAccessException
        || (e instanceof HttpServerErrorException h && h.getStatusCode().value() == 503))
    .build();
Retry retry = Retry.of("inventory", retryConfig);

var factory = new JdkClientHttpRequestFactory(HttpClient.newBuilder().connectTimeout(Duration.ofMillis(300)).build());
factory.setReadTimeout(Duration.ofMillis(800));
RestClient client = RestClient.builder().requestFactory(factory).baseUrl("http://inventory").build();

Stock stock = Retry.decorateSupplier(retry,
    () -> client.get().uri("/stock/{sku}", sku).retrieve().body(Stock.class)).get();

Retry storms and budgets

If each of three layers retries three times, the bottom service receives up to 27 attempts per user request during an outage. Mitigations: retry at one layer only, retry budgets (e.g. retries may add at most 10% extra load, a technique used in Finagle and Envoy), and circuit breakers to stop retrying a dead dependency.

Pros and cons

Pros Cons
Masks transient failures Amplifies load during outages if uncontrolled
Simple to add Duplicate side effects without idempotency
Timeouts bound resource usage Wrong timeouts cause false failures or slow failure

Interview questions

Should you retry a POST that creates a payment?

Only with an idempotency key that the server honours, so a retry returns the original result instead of creating a second payment. Without it, query status first instead of retrying.

How do you choose a timeout value?

From the dependency's measured latency distribution (e.g. slightly above p99.9) and the caller's total latency budget, then validate in load tests. Revisit when dependencies change.