The business wants the checkout to survive a full cloud region outage with no data loss. What are the options and trade-offs?
Short answer
RPO zero across regions requires synchronous cross-region commits (consensus-based databases like Spanner or Aurora DSQL, or synchronous replication), which adds write latency. Alternatives with small data loss (async replication, warm standby) are far cheaper. Quantify downtime cost, choose per capability, and test failover regularly.
Detailed answer
| Option | RPO | RTO | Cost/complexity | Notes |
|---|---|---|---|---|
| Warm standby, async replication | Seconds | Minutes | Medium | Most common; small data-loss window |
| Active-active, async multi-writer (DynamoDB global tables, Cosmos DB multi-write) | Seconds; conflicts possible | ~0 | High | Needs conflict-tolerant data model |
| Active-active, synchronous consensus (Spanner multi-region, Aurora DSQL, CockroachDB) | 0 | ~0 | High | Write latency includes cross-region quorum |
Steps: define RPO/RTO per capability (payments may need 0; cart can lose seconds); keep idempotency keys so retried orders after failover are not duplicated; ensure all dependencies (identity, secrets, PSP connectivity, DNS) are multi-region; automate and rehearse failover.
Architecture diagram
flowchart LR GLB[Global load balancer] --> RA[Region A checkout] GLB --> RB[Region B checkout] RA --> DB[(Multi-region consensus DB<br/>quorum across 3 regions)] RB --> DB
Real-world example
Google Spanner's multi-region configurations offer up to 99.999% availability with strongly consistent writes at the cost of higher write latency than regional configurations.
Follow-up questions
- What about the payment provider during failover? (Multiple PSP endpoints/regions; idempotent retries.)
- How do you avoid split brain? (Consensus quorum across at least three regions or a witness.)
What the interviewer expects
Understanding that RPO 0 has a latency and cost price, nuanced per-capability choices, and attention to dependencies and testing.
Common mistakes to avoid
Claiming RPO 0 with asynchronous replication, forgetting dependencies, never testing failover.