GitOps
ArgoCD, Flux, Jenkins X - declarative continuous delivery where Git is the source of truth for what runs in production
GitOps
GitOps is a deployment model where the desired state of your infrastructure lives in Git, and an operator continuously reconciles the running cluster against it. You don't push to production; you push to Git, and the cluster pulls.
The term was coined by Weaveworks in 2017 for Kubernetes deployment, but the pattern works for anything declarative: Crossplane manages cloud resources via GitOps, Atlas does database schemas, Karavel does platform configurations.
Why GitOps
| Without GitOps | With GitOps |
|---|---|
kubectl apply from a CI runner | Cluster reconciles itself from Git |
| What's running ≠ what's in Git (drift) | Git is the truth; drift is auto-corrected |
| Audit "what changed" via Slack archaeology | Git log is the audit trail |
| Rollback = redeploy from previous CI job | Rollback = git revert |
| Credentials for prod live in CI runners | CI never touches prod; pull model |
| Manual cluster changes silently linger | Detected and reverted |
| Multi-cluster deploys are fragile scripts | One Git repo, N clusters, each pulls |
The key insight: push-based CD ("CI runner pushes to cluster") gives the CI system production credentials. Pull-based CD ("cluster pulls from Git") doesn't. Security and operability both improve.
The Core Loop
┌───────────────────────────────────────────────┐
│ Git repo │
│ ┌──────────────────────────────────────┐ │
│ │ apps/checkout/deployment.yaml │ │
│ │ apps/checkout/service.yaml │ │
│ │ infrastructure/cluster-config.yaml │ │
│ └──────────────────────────────────────┘ │
└───────────┬───────────────────────────────────┘
│ poll / webhook every 1-3 min
▼
┌───────────────────────────────────────────────┐
│ GitOps operator (in cluster) │
│ - Diff Git state vs. cluster state │
│ - Apply differences │
│ - Report status (synced / out-of-sync) │
└───────────┬───────────────────────────────────┘
│
▼
┌───────────────────────────────────────────────┐
│ Kubernetes cluster │
└───────────────────────────────────────────────┘The loop never stops. Manual kubectl edit drifts the cluster; the next reconcile snaps it back. Convergence is enforced, not optional.
The Players
Kubernetes GitOps
| Tool | Maturity | Best for |
|---|---|---|
| ArgoCD | CNCF graduated | Most popular; UI-first; multi-cluster; sync waves |
| Flux | CNCF graduated | CLI-first; multi-tenancy strong; OCI sources |
| Jenkins X | CNCF incubating | Built around GitOps for the full pipeline |
| Rancher Fleet | Stable | Massive scale (thousands of clusters); SUSE-backed |
| Weave GitOps | OSS (was Weaveworks) | Less active after Weaveworks closed |
ArgoCD vs Flux is the modern question. Both excellent:
| ArgoCD | Flux | |
|---|---|---|
| UI | Rich web UI | None (CLI / API only) |
| Helm | First-class | First-class |
| Kustomize | First-class | First-class |
| Multi-tenancy | Project-based, RBAC | Native multi-tenancy (tenant CRDs) |
| Image automation | Argo Image Updater (add-on) | Built-in image automation controller |
| Sync waves | Yes | Via Kustomize dependencies |
| OCI artifacts | Yes (v2.7+) | Yes (native, earlier) |
| Notifications | Native | Native + Provider CRDs |
Default recommendation: ArgoCD for teams that want a UI and have multiple environments to navigate. Flux for platform teams who want everything in YAML.
Beyond Kubernetes
- Crossplane + ArgoCD → manage cloud resources (RDS, S3, IAM) declaratively in Git
- Atlas → GitOps for database schemas
- Terraform + Atlantis → Terraform's closest GitOps cousin (PR-driven Terraform)
- Pulumi GitOps → IaC with GitOps loop
Repository Patterns
Three common structures:
Mono-repo (app + config)
repo/
├── src/ # application code
├── manifests/ # k8s manifests for THIS app
│ ├── base/
│ └── overlays/
│ ├── staging/
│ └── prod/
└── .github/workflows/ # CI builds image, updates manifestsSimple. Works for single-team services.
Config repo separate from app repo
app-checkout/ # one per app
└── src/
infra-config/ # the GitOps repo
├── apps/
│ ├── checkout/
│ │ ├── deployment.yaml
│ │ └── kustomization.yaml
│ └── api/
└── clusters/
├── staging/
└── prod/Recommended at any real scale. App teams own their app repo; platform team curates the config repo (or apps PR into it).
Repo-per-tenant
team-payments-config/
team-growth-config/
platform-config/For large orgs with strong team autonomy. ArgoCD's ApplicationSet or Flux's GitRepository per repo handles this.
Promotion: How Code Reaches Prod
The deploy event is "image tag X is now referenced in Git under the prod overlay." How that happens:
- Image automation: Flux Image Update Controller / Argo Image Updater watches the registry; when a new image matches a regex, opens a PR / commits to bump.
- CI pushes the bump: build pipeline updates the Git ref after image push.
- Manual promotion: human edits the manifest, opens a PR. Used for prod with sensitive workloads.
A common production setup:
- staging: image automation (auto-deploys on every build)
- prod: PR-based promotion with approval (or canary controller)
What Lives in Git
Beyond app manifests, GitOps holds:
- Cluster bootstrap: ArgoCD itself, Cilium, ingress controllers
- Namespace + RBAC: who can do what, in code
- Secrets (encrypted): Sealed Secrets, SOPS, External Secrets Operator
- Cloud resources: via Crossplane / Atlas
- Network policies, PodSecurityPolicy, OPA constraints
- Service mesh config: Istio VirtualServices, Linkerd ServiceProfiles
- Observability config: Prometheus rules, Grafana dashboards-as-code
Everything reconcilable belongs in Git. Stateful data (databases, object storage) doesn't — that's what backups are for.
Learning Path
1. Getting Started
Install ArgoCD on kind; bootstrap from a Git repo; deploy an app; watch reconciliation; rollback via git revert
2. Patterns
App-of-apps, ApplicationSets, promotion workflows, secret management, multi-cluster, drift detection
3. Best Practices
Repo structure, branch strategy, RBAC, disaster recovery, common pitfalls, scaling GitOps to many teams
When GitOps Doesn't Fit
Honest cases:
- You don't have Kubernetes. The pattern works elsewhere (Terraform GitOps, Atlas for SQL) but ArgoCD/Flux are K8s-shaped.
- Single developer, single small service. Push-based is fine; GitOps is overhead.
- Workloads aren't declarative. A side-effecting deploy script GitOps poorly.
- You need imperative pre/post hooks. Possible (Argo
PreSync/PostSync) but more friction than pipelines. - Compliance demands separation between change author and deployer. GitOps + PR review + branch protection covers most regimes, but some demand a separate deploy team — fits poorly.
The biggest GitOps mistake: treating it as a tool, not a discipline. Installing ArgoCD doesn't make you "doing GitOps" — what makes you doing GitOps is the rule that nothing reaches production except via Git. The first time someone runs kubectl apply in prod, the discipline breaks. Enforce it (no direct cluster access in prod) and the pattern delivers.