Steven's Knowledge

GitOps

ArgoCD, Flux, Jenkins X - declarative continuous delivery where Git is the source of truth for what runs in production

GitOps

GitOps is a deployment model where the desired state of your infrastructure lives in Git, and an operator continuously reconciles the running cluster against it. You don't push to production; you push to Git, and the cluster pulls.

The term was coined by Weaveworks in 2017 for Kubernetes deployment, but the pattern works for anything declarative: Crossplane manages cloud resources via GitOps, Atlas does database schemas, Karavel does platform configurations.

Why GitOps

Without GitOpsWith GitOps
kubectl apply from a CI runnerCluster reconciles itself from Git
What's running ≠ what's in Git (drift)Git is the truth; drift is auto-corrected
Audit "what changed" via Slack archaeologyGit log is the audit trail
Rollback = redeploy from previous CI jobRollback = git revert
Credentials for prod live in CI runnersCI never touches prod; pull model
Manual cluster changes silently lingerDetected and reverted
Multi-cluster deploys are fragile scriptsOne Git repo, N clusters, each pulls

The key insight: push-based CD ("CI runner pushes to cluster") gives the CI system production credentials. Pull-based CD ("cluster pulls from Git") doesn't. Security and operability both improve.

The Core Loop

┌───────────────────────────────────────────────┐
│                   Git repo                    │
│  ┌──────────────────────────────────────┐     │
│  │ apps/checkout/deployment.yaml        │     │
│  │ apps/checkout/service.yaml           │     │
│  │ infrastructure/cluster-config.yaml   │     │
│  └──────────────────────────────────────┘     │
└───────────┬───────────────────────────────────┘
            │ poll / webhook every 1-3 min

┌───────────────────────────────────────────────┐
│           GitOps operator (in cluster)        │
│   - Diff Git state vs. cluster state          │
│   - Apply differences                         │
│   - Report status (synced / out-of-sync)      │
└───────────┬───────────────────────────────────┘


┌───────────────────────────────────────────────┐
│              Kubernetes cluster               │
└───────────────────────────────────────────────┘

The loop never stops. Manual kubectl edit drifts the cluster; the next reconcile snaps it back. Convergence is enforced, not optional.

The Players

Kubernetes GitOps

ToolMaturityBest for
ArgoCDCNCF graduatedMost popular; UI-first; multi-cluster; sync waves
FluxCNCF graduatedCLI-first; multi-tenancy strong; OCI sources
Jenkins XCNCF incubatingBuilt around GitOps for the full pipeline
Rancher FleetStableMassive scale (thousands of clusters); SUSE-backed
Weave GitOpsOSS (was Weaveworks)Less active after Weaveworks closed

ArgoCD vs Flux is the modern question. Both excellent:

ArgoCDFlux
UIRich web UINone (CLI / API only)
HelmFirst-classFirst-class
KustomizeFirst-classFirst-class
Multi-tenancyProject-based, RBACNative multi-tenancy (tenant CRDs)
Image automationArgo Image Updater (add-on)Built-in image automation controller
Sync wavesYesVia Kustomize dependencies
OCI artifactsYes (v2.7+)Yes (native, earlier)
NotificationsNativeNative + Provider CRDs

Default recommendation: ArgoCD for teams that want a UI and have multiple environments to navigate. Flux for platform teams who want everything in YAML.

Beyond Kubernetes

  • Crossplane + ArgoCD → manage cloud resources (RDS, S3, IAM) declaratively in Git
  • Atlas → GitOps for database schemas
  • Terraform + Atlantis → Terraform's closest GitOps cousin (PR-driven Terraform)
  • Pulumi GitOps → IaC with GitOps loop

Repository Patterns

Three common structures:

Mono-repo (app + config)

repo/
├── src/                    # application code
├── manifests/              # k8s manifests for THIS app
│   ├── base/
│   └── overlays/
│       ├── staging/
│       └── prod/
└── .github/workflows/      # CI builds image, updates manifests

Simple. Works for single-team services.

Config repo separate from app repo

app-checkout/               # one per app
└── src/

infra-config/               # the GitOps repo
├── apps/
│   ├── checkout/
│   │   ├── deployment.yaml
│   │   └── kustomization.yaml
│   └── api/
└── clusters/
    ├── staging/
    └── prod/

Recommended at any real scale. App teams own their app repo; platform team curates the config repo (or apps PR into it).

Repo-per-tenant

team-payments-config/
team-growth-config/
platform-config/

For large orgs with strong team autonomy. ArgoCD's ApplicationSet or Flux's GitRepository per repo handles this.

Promotion: How Code Reaches Prod

The deploy event is "image tag X is now referenced in Git under the prod overlay." How that happens:

  1. Image automation: Flux Image Update Controller / Argo Image Updater watches the registry; when a new image matches a regex, opens a PR / commits to bump.
  2. CI pushes the bump: build pipeline updates the Git ref after image push.
  3. Manual promotion: human edits the manifest, opens a PR. Used for prod with sensitive workloads.

A common production setup:

  • staging: image automation (auto-deploys on every build)
  • prod: PR-based promotion with approval (or canary controller)

What Lives in Git

Beyond app manifests, GitOps holds:

  • Cluster bootstrap: ArgoCD itself, Cilium, ingress controllers
  • Namespace + RBAC: who can do what, in code
  • Secrets (encrypted): Sealed Secrets, SOPS, External Secrets Operator
  • Cloud resources: via Crossplane / Atlas
  • Network policies, PodSecurityPolicy, OPA constraints
  • Service mesh config: Istio VirtualServices, Linkerd ServiceProfiles
  • Observability config: Prometheus rules, Grafana dashboards-as-code

Everything reconcilable belongs in Git. Stateful data (databases, object storage) doesn't — that's what backups are for.

Learning Path

When GitOps Doesn't Fit

Honest cases:

  • You don't have Kubernetes. The pattern works elsewhere (Terraform GitOps, Atlas for SQL) but ArgoCD/Flux are K8s-shaped.
  • Single developer, single small service. Push-based is fine; GitOps is overhead.
  • Workloads aren't declarative. A side-effecting deploy script GitOps poorly.
  • You need imperative pre/post hooks. Possible (Argo PreSync/PostSync) but more friction than pipelines.
  • Compliance demands separation between change author and deployer. GitOps + PR review + branch protection covers most regimes, but some demand a separate deploy team — fits poorly.

The biggest GitOps mistake: treating it as a tool, not a discipline. Installing ArgoCD doesn't make you "doing GitOps" — what makes you doing GitOps is the rule that nothing reaches production except via Git. The first time someone runs kubectl apply in prod, the discipline breaks. Enforce it (no direct cluster access in prod) and the pattern delivers.

On this page