The business wants the checkout to survive a full cloud region outage with no data loss. What are the options and trade-offs?

Short answer

RPO zero across regions requires synchronous cross-region commits (consensus-based databases like Spanner or Aurora DSQL, or synchronous replication), which adds write latency. Alternatives with small data loss (async replication, warm standby) are far cheaper. Quantify downtime cost, choose per capability, and test failover regularly.

Detailed answer

Option RPO RTO Cost/complexity Notes
Warm standby, async replication Seconds Minutes Medium Most common; small data-loss window
Active-active, async multi-writer (DynamoDB global tables, Cosmos DB multi-write) Seconds; conflicts possible ~0 High Needs conflict-tolerant data model
Active-active, synchronous consensus (Spanner multi-region, Aurora DSQL, CockroachDB) 0 ~0 High Write latency includes cross-region quorum

Steps: define RPO/RTO per capability (payments may need 0; cart can lose seconds); keep idempotency keys so retried orders after failover are not duplicated; ensure all dependencies (identity, secrets, PSP connectivity, DNS) are multi-region; automate and rehearse failover.

Architecture diagram

flowchart LR
  GLB[Global load balancer] --> RA[Region A checkout]
  GLB --> RB[Region B checkout]
  RA --> DB[(Multi-region consensus DB<br/>quorum across 3 regions)]
  RB --> DB

Real-world example

Google Spanner's multi-region configurations offer up to 99.999% availability with strongly consistent writes at the cost of higher write latency than regional configurations.

Follow-up questions

  • What about the payment provider during failover? (Multiple PSP endpoints/regions; idempotent retries.)
  • How do you avoid split brain? (Consensus quorum across at least three regions or a witness.)

What the interviewer expects

Understanding that RPO 0 has a latency and cost price, nuanced per-capability choices, and attention to dependencies and testing.

Common mistakes to avoid

Claiming RPO 0 with asynchronous replication, forgetting dependencies, never testing failover.