Steven's Knowledge

Getting Started

Install OPA Gatekeeper and Kyverno on kind, write a first policy, block a bad pod, run conftest on Terraform

Getting Started

This page installs both OPA Gatekeeper and Kyverno on a local cluster, writes one policy in each, sees them block bad workloads, then uses conftest to check Terraform code.

Prerequisites

kind create cluster --name policy
brew install opa conftest    # or download from openpolicyagent.org

Path A: OPA Gatekeeper

Gatekeeper is OPA packaged as a Kubernetes admission controller, with ConstraintTemplate and Constraint CRDs.

Install

kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/release-3.16/deploy/gatekeeper.yaml

kubectl wait --for=condition=available --timeout=300s \
  -n gatekeeper-system deployment/gatekeeper-controller-manager

Define a policy: required label

A ConstraintTemplate defines a kind of policy (parameterized):

# template-required-labels.yaml
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names: { kind: K8sRequiredLabels }
      validation:
        openAPIV3Schema:
          type: object
          properties:
            labels:
              type: array
              items: { type: string }
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels
        violation[{"msg": msg}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("missing required labels: %v", [missing])
        }

Apply, then create a Constraint (the policy instance):

# constraint-must-have-owner.yaml
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-must-have-owner
spec:
  match:
    kinds:
      - { apiGroups: [""], kinds: ["Namespace"] }
  parameters:
    labels: ["owner"]
kubectl apply -f template-required-labels.yaml
kubectl apply -f constraint-must-have-owner.yaml

See it work

# This fails: no owner label
kubectl create namespace test-ns
# Error from server: admission webhook "validation.gatekeeper.sh" denied the request

# This succeeds
kubectl create namespace test-ns --dry-run=client -o yaml | \
  kubectl label --local -f - owner=alice -o yaml | \
  kubectl apply -f -

The policy is enforced at the API server before the resource exists. No bad resource gets created.

Path B: Kyverno

Same goal, different style: policy as YAML.

Install

helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
helm install kyverno kyverno/kyverno -n kyverno --create-namespace

Define a policy: deny root containers

# disallow-root.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: disallow-root-user
spec:
  validationFailureAction: Enforce
  rules:
    - name: validate-runAsNonRoot
      match:
        any:
          - resources: { kinds: [Pod] }
      validate:
        message: "Containers must not run as root."
        pattern:
          spec:
            containers:
              - securityContext:
                  runAsNonRoot: true
kubectl apply -f disallow-root.yaml

See it work

# This fails (no securityContext)
kubectl run nginx --image=nginx
# Error: ... validation error: Containers must not run as root.

# This succeeds
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata: { name: nginx, namespace: default }
spec:
  containers:
    - name: nginx
      image: nginx
      securityContext: { runAsNonRoot: true, runAsUser: 1000 }
EOF

Mutation: Add Defaults Automatically

A nicer pattern: instead of rejecting, add the missing thing. Kyverno mutate policy:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata: { name: add-default-owner }
spec:
  rules:
    - name: add-owner-label
      match:
        any: [{ resources: { kinds: [Namespace] } }]
      mutate:
        patchStrategicMerge:
          metadata:
            labels:
              owner: unknown

Now namespaces without owner get owner=unknown automatically — a soft enforcement that you can later upgrade to hard (validating) once everyone is using real labels.

Path C: Terraform Pre-Plan with conftest

Policies don't have to be runtime. Catch them in CI before resources exist.

Sample Terraform

# main.tf
resource "aws_s3_bucket" "data" {
  bucket = "my-company-data"
  acl    = "public-read"   # Bad
}

Plan to JSON

terraform init
terraform plan -out=tfplan.binary
terraform show -json tfplan.binary > tfplan.json

Policy

# policy/s3.rego
package main

deny[msg] {
  resource := input.resource_changes[_]
  resource.type == "aws_s3_bucket"
  resource.change.after.acl == "public-read"
  msg := sprintf("S3 bucket %v has public ACL", [resource.address])
}

Run conftest

conftest test --policy policy/ tfplan.json
# FAIL - tfplan.json - main - S3 bucket aws_s3_bucket.data has public ACL
# 1 test, 0 passed, 1 failure

Wire this into CI (conftest test in your GitHub Actions job) and bad Terraform never merges.

Path D: Library Policies

You don't have to write everything. Use the libraries:

# CIS Kubernetes Benchmark policies for Gatekeeper
kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper-library/master/library/general/...

# Kyverno policies (CIS + best practices)
kubectl apply -f https://raw.githubusercontent.com/kyverno/policies/release-1.13/...

# Conftest policies for Terraform/Kubernetes/CI
git clone https://github.com/open-policy-agent/conftest

For Kubernetes especially, ~80% of the policies you want already exist as battle-tested libraries.

Cleanup

kind delete cluster --name policy

What's Next

  • Patterns — bundles, testing, mutation, exceptions, rollout, library structure
  • Best Practices — lifecycle, performance, debugging, compliance, pitfalls

On this page