Best Practices

Object storage is "boring infrastructure" until you wake up to a public-bucket leak, a five-figure egress bill, or a deleted bucket. These habits keep it boring.

Security: Don't Make a Public Bucket Accident

The #1 cloud security incident is "bucket left public." Defenses:

For new accounts: turn on BPA at the account level, set Object Ownership to "Bucket Owner Enforced" on every bucket, deny * Principal in bucket policies. Static-site hosting goes through a CDN with origin auth — the bucket itself stays private.

IAM: Least Privilege

Per-object-storage roles, not "S3 full access":

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::user-uploads/*"
    },
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::user-uploads",
      "Condition": {
        "StringLike": { "s3:prefix": "uploads/${aws:userid}/*" }
      }
    }
  ]
}

For CI/CD and runtime workloads, use OIDC to assume short-lived roles — never long-lived access keys (see Secrets Management).

Encryption

At Rest

Always on; the question is which key:

For most workloads: SSE-KMS with a customer-managed key. Auditable via CloudTrail; key access can be revoked.

In Transit

Always HTTPS. Bucket policy can enforce it:

{
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:*",
  "Resource": ["arn:aws:s3:::mybucket", "arn:aws:s3:::mybucket/*"],
  "Condition": { "Bool": { "aws:SecureTransport": "false" } }
}

Bucket Naming

A naming convention you don't have to think about:

<org>-<env>-<purpose>-<region>
example-prod-uploads-us-east-1
example-staging-logs-eu-west-1
example-prod-backups-ap-southeast-2

• Org prefix because S3 bucket names are globally unique.
• Env in name prevents staging/prod confusion at a glance.
• Purpose so logs aren't mixed with user data.
• Region suffix so multi-region setups are obvious.

Avoid dots in bucket names — they cause TLS issues with virtual-hosted URLs. Lowercase, hyphens only.

Object Key Design

Keys aren't directories, but they look like them. Two principles:

Don't put hot prefixes at the front

Old S3 partitioned by key prefix. Modern S3 (2018+) auto-partitions and this matters less, but for R2 and other providers, hot prefixes still matter. Pattern:

Bad:   2025-05-21/user-42/event-...     (everyone writes to today's prefix)
OK:    user-42/2025-05-21/event-...     (writes spread across users)
Best:  <hash-prefix>/user-42/...        (truly even distribution)

For uploads from many users, putting user-id first naturally spreads load.

Make keys deterministic where it helps

For idempotent uploads, derive the key from content (sha256:abc...) — same input always produces the same key, accidental re-uploads are no-ops. For per-user uploads, prefix with user-id so per-user IAM policies are easy.

Cost Control

Object storage looks cheap but bills surprise. Watch:

Optimization checklist

• Lifecycle to IA/Glacier for cold data.
• Compress before upload if applicable (gzip text, webp images).
• CloudFront / CDN in front to avoid egress on cache hits.
• Use R2 if you have heavy distribution traffic.
• Abort incomplete multipart uploads (lifecycle rule).
• Delete old versions after a retention window.
• Right-size storage class — STANDARD for everything is rarely optimal.

For AWS, Storage Lens and Cost Explorer show the breakdown. R2 has its dashboard. Audit monthly.

Observability

Most providers ship metrics to their monitoring service (CloudWatch, Cloudflare Analytics). Pipe interesting ones into Prometheus & Grafana for cross-cutting dashboards.

Server access logging (S3 server access logs, R2's analytics) goes to another bucket. Useful for security audits and debugging.

Common Pitfalls

Compliance and Object Lock

For regulated data (PCI, HIPAA, financial records):

• Object Lock in Compliance mode — objects can't be deleted before retention period, even by root. Use for audit trails and financial records.
• Object Lock in Governance mode — admins with the right permission can delete; default for "important but not legally locked."
• Bucket policies that deny delete without MFA.
• Audit logs (CloudTrail / equivalent) shipped off-account.

Multi-Provider Strategy

For redundancy across providers:

• Primary bucket on R2 for cheap egress.
• Replicated to S3 for AWS-side integrations.
• Cold-copy to Backblaze quarterly for DR / cost.

Tools: rclone for ad-hoc sync, Cyberduck / s3-mirror / various pipelines for production. Storage cost roughly doubles; some peace of mind.

Checklist