Object Storage
S3-compatible blob storage - AWS S3, Cloudflare R2, GCS, Azure Blob, MinIO - the backbone of modern data persistence
Object Storage
Object storage holds unstructured blobs — files, images, video, backups, log archives, ML training data — at effectively unlimited scale, accessed over HTTP. It's the workhorse behind almost every modern system: every CDN origin, every backup target, every data lake, every user-uploaded photo.
What It Is (and Isn't)
| Object storage | Block storage | File system |
|---|---|---|
| HTTP API, immutable blobs | Mountable disk | POSIX semantics |
| Cheap per GB; high latency per op | Fast per op; tied to one VM | Per-file ops; tied to a server or share |
| Eventually consistent (historically) / strong now | Strong | Strong |
| Scales to petabytes | TB per volume | Server-bound |
| Examples: S3, R2, GCS | EBS, GCE PD | EFS, NFS |
If you find yourself thinking "S3 with directories" — object storage simulates directories via key prefixes (photos/2025/05/...), but the underlying model is a flat key-value store. Operations are per-object, not per-tree.
Why Object Storage
| Without | With |
|---|---|
| Files on a single VM disk — single point of failure | 99.999999999% (11 nines) durability across zones |
| Database BLOB columns — bloats DB, slows backups | Database stores URLs; storage handles blobs |
| Local disk shared across services via NFS | Every service hits storage directly via HTTPS |
| Manual backup of files | Lifecycle rules move old data to cheap tiers automatically |
| Serving uploads from your app server | Browser uploads directly via presigned URLs |
The Players
| Provider | Notes |
|---|---|
| AWS S3 | The original; the API everyone copies. Pricing per GB + per-request + egress |
| Cloudflare R2 | S3-compatible API; zero egress fees — disruptive for distribution-heavy workloads |
| Google Cloud Storage (GCS) | First-class GCP integration; native multi-region |
| Azure Blob Storage | Microsoft equivalent; deep hot/cool/archive tiering |
| Backblaze B2 | Very cheap; S3-compatible; less feature-rich |
| Wasabi | Same niche — cheap, S3-compatible, hosted |
| MinIO | Open-source, self-hostable; S3 API; popular for on-prem / hybrid |
| SeaweedFS / Ceph RGW / Garage | Open-source for hyperscale or air-gapped |
| DigitalOcean Spaces / Linode Object Storage | S3-compatible, simpler pricing |
The S3 API is the de facto standard — most clients work against any S3-compatible storage by changing the endpoint URL.
Choosing a Provider
A short decision tree:
| You need... | Pick |
|---|---|
| The most-supported API and richest ecosystem | AWS S3 |
| Public distribution at scale (CDN origin, downloads) | R2 — egress is free |
| Tight GCP / BigQuery integration | GCS |
| Tight Azure integration | Azure Blob |
| Cheapest cold storage | Backblaze B2 / Wasabi |
| Self-host / air-gap / on-prem | MinIO |
| Multi-cloud abstraction | All-of-the-above via S3-compatible clients |
For new projects in 2026: R2 if you serve significant traffic (CDN savings dominate), S3 if you're already on AWS (no egress to AWS services from S3).
Learning Path
1. Getting Started
Run MinIO locally, talk to it with the AWS CLI and Node SDK, upload and download
2. Patterns
Presigned URLs, multipart upload, lifecycle, versioning, static hosting, replication
3. Best Practices
Security (bucket policies, encryption), cost, observability, naming, common pitfalls
What's a Bucket, an Object, a Key
s3://my-bucket/photos/2025/05/sunset.jpg
│ │ │
│ └─ bucket └─ key (the full path within the bucket)
└─ scheme- Bucket — top-level container. Globally named (S3) or per-account named (R2). Has a region, policy, lifecycle rules.
- Object — the actual blob: bytes + metadata + headers.
- Key — the object's identifier within the bucket. Looks like a path but is a flat string.
- Metadata — HTTP-style headers (
Content-Type,Cache-Control) + user-defined (x-amz-meta-*).
Operations are per-object. There's no "rename a directory" because there are no directories.
Object storage is immutable per object. Writing to photos/sunset.jpg either creates a new version (if versioning is on) or overwrites — you can't append, you can't partial-update. For mutable structured data you want a database; object storage is for blobs.