Disaster Recovery & Backup

Disaster recovery (DR) is the discipline of getting back online after things go very wrong: a region outage, an accidentally-dropped database, a ransomware encryption of your storage, a deletion that wasn't supposed to happen. Backup is one tool in the DR toolbox; DR also includes replicas, failover patterns, runbooks, and the practice of testing it all.

The hard truth: untested backups don't exist. Every team has a backup process; far fewer have actually restored from one. The first time you find out your backup is incomplete or unrestorable is during the worst day of your career.

Why DR Deliberately

The cost of DR isn't huge if you build it in early; it's catastrophic to retrofit during an incident.

RTO and RPO

Two numbers define the recovery target:

Stronger numbers cost more. Match RTO/RPO to business impact per service tier:

A tier-1 service costs a lot more to operate than a tier-3 one. Don't apply tier-1 expectations to everything; you'll burn money on the unimportant.

The Players

Kubernetes-native backup

Block / file backup

Database backup

Each engine has native tools:

• Postgres: pg_basebackup, WAL archiving (PITR), logical replication
• MySQL: mysqldump, xtrabackup, binlog-based PITR
• MongoDB: mongodump, oplog tailing
• Cassandra: nodetool snapshot + sstableloader
• DynamoDB / RDS: AWS-managed PITR, snapshots
• Elasticsearch: _snapshot API to S3

Cloud DR services

Managed services like RDS, Cloud SQL, DynamoDB make backup and PITR a checkbox — use them.

The Backup Bestiary

Not all backups are equal:

Most production stacks combine these: snapshots for fast recovery, replication for low RPO, PITR for fine-grained recovery, immutable offsite for ransomware.

The 3-2-1 Rule

Time-tested guidance: 3 copies of data, on 2 different media, with 1 offsite.

Modern interpretation:

• 3 copies: primary, secondary (replica or snapshot), tertiary (backup)
• 2 different media/services: e.g., live storage + S3 + cold archive
• 1 offsite (different region or different cloud)
• Modern addition: at least 1 is immutable

Why offsite: a region outage takes your primary and your in-region backup. Why immutable: ransomware encrypts everything it can reach.

Learning Path

Multi-Region: The Spectrum

How "multi-region" you go is a slider, not a switch:

single region
    │
    ▼
single region + cross-region backup       (RPO: hours)
    │
    ▼
single region + cross-region warm standby  (RPO: minutes; RTO: minutes-hours)
    │
    ▼
active-passive multi-region                (RPO: seconds; RTO: minutes)
    │
    ▼
active-active multi-region                 (RPO: zero; RTO: ~zero)
    │
    ▼
multi-cloud + multi-region                  (sovereignty / vendor)

Each step is more complex and more expensive. The right answer is rarely the most rigorous — match to actual business impact and budget.

Backup Anti-Patterns

No restore test. Backup process runs nightly; never restored. You don't have backups, you have hope. Test restores monthly minimum.

Backups on the same volume as data. The fire that destroyed your data destroys the backup. Offsite — different region, different cloud, different physical location.

Backups without immutability. Ransomware encrypts the backup along with the source. Use S3 Object Lock, Glacier Vault Lock, or air-gapped storage.

Backup the data, not the schema. Restoring rows is useless without the database schema, the app version that wrote them, and the config that made them mean what they meant.

Forgetting secrets. Restoring the database is great. Without the encryption key for the data at rest, the restored disk is useless. Back up the system, including all its dependencies.

Tying backup to a vendor that can be locked out. Your account gets suspended; your backups go too. Back up out of any single vendor's domain.