M7.5 — Helm & Kustomize Primer: one manifest, many environments¶

Core question: Your app needs to run in dev, staging, and prod — same Deployment, but different replicas, image tags, and resource limits. Copy-pasting three YAML folders rots in a week. What's the production answer?

⏱️ Time: ~25 min padho + 20 min lab · 🎚️ Level: Beginner-friendly · 📋 Pehle chahiye: M4 (plain YAML likh chuke ho), M7 (Argo ne "Helm/Kustomize render" mention kiya — yeh usi ka follow-up hai)

Is module ke baad tum kar paoge: - Helm chart install/upgrade/rollback karo aur values.yaml se environments alag karo - Kustomize base + overlay bana kar ek hi YAML se dev/prod nikालो - Interview me ek line me bolo: Helm kab, Kustomize kab — aur kyun

Cross-links: M7 GitOps said Argo "renders manifests (raw YAML, Helm, Kustomize)" — this is what that meant. ch19 uses kustomize edit set image in CI; ch28 goes deep on Helm with your two real projects.

The 60-second version¶

Both tools solve one problem: YAML duplication across environments. They just pick opposite strategies:

Helm = fill-in-the-blanks. Manifests become templates with placeholders ({{ .Values.replicas }}); each environment supplies its own answers file (values.yaml). Bonus: versioned install/upgrade/rollback of the whole bundle (a release).
Kustomize = photocopy-plus-sticky-notes. Keep one plain-YAML base; each environment is an overlay — a small patch file listing only what differs. No templates, no new syntax.

🇮🇳 Hinglish intuition: Helm = shaadi ka card printer — ek design, blanks me naam/date/venue har party ke liye alag bhar do. Kustomize = original letter + sticky notes — letter mat chhedo, note chipka do "prod me replicas 5 kar dena."

The problem, concretely (why copy-paste dies)¶

flowchart TD
    subgraph BAD["❌ Copy-paste approach"]
        D1["dev/deployment.yaml<br/>replicas: 1"]
        D2["staging/deployment.yaml<br/>replicas: 2"]
        D3["prod/deployment.yaml<br/>replicas: 5"]
        FIX["security fix needed in<br/>the Deployment spec"]:::warn
        FIX -->|"edit"| D1
        FIX -->|"edit"| D2
        FIX -->|"edit... forgot? 💥"| D3
    end

    subgraph GOOD["✅ Base + variation approach (both tools)"]
        BASE["ONE source of truth<br/>(template or base)"]:::run
        V1["dev variation<br/>(values / overlay)"]
        V2["staging variation"]
        V3["prod variation"]
        BASE --> V1 & V2 & V3
        FIX2["security fix"]:::run
        FIX2 -->|"edit ONCE"| BASE
    end

    classDef warn fill:#fdeeee,stroke:#d64545,color:#b23030;
    classDef run fill:#e0f2f1,stroke:#00897b,color:#004d40;

Three copies = three chances to forget one. One base + small variations = fix once, everywhere. This diagram IS the interview answer for "why Helm/Kustomize?"

Helm in 10 minutes¶

The three ideas¶

Idea	What it is	Analogy
Chart	A folder of templated manifests + metadata — the installable package	Shaadi-card ka design
values.yaml	The answers file that fills the template's blanks	Naam/date/venue ki list
Release	One installed instance of a chart, with version history	Ek party ke chhape hue cards

What a chart looks like¶

mychart/
├── Chart.yaml            # name + version of the package
├── values.yaml           # DEFAULT answers (every blank ka default)
└── templates/
    ├── deployment.yaml   # manifest with blanks
    └── service.yaml

# templates/deployment.yaml (the blanks)
spec:
  replicas: {{ .Values.replicaCount }}          # ← blank
  template:
    spec:
      containers:
        - name: app
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"   # ← blanks

# values.yaml (the default answers)
replicaCount: 1
image:
  repository: nginx
  tag: "1.27"

# prod gets different answers WITHOUT touching the template:
helm install myapp ./mychart -f values-prod.yaml     # values-prod.yaml: replicaCount: 5

The commands that matter (90% of real usage)¶

helm template myapp ./mychart          # render locally, SEE the final YAML — always do this first
helm install myapp ./mychart          # first deploy (creates release "myapp")
helm upgrade myapp ./mychart          # apply changes (new release revision)
helm upgrade --install myapp ./mychart # CI favourite: install if absent, upgrade if present
helm rollback myapp 1                 # back to revision 1 — one command
helm history myapp                    # every revision = audit trail
helm repo add bitnami https://charts.bitnami.com/bitnami   # use others' charts
helm install db bitnami/postgresql    # battle-tested Postgres in one line

The superpower nobody mentions first: OTHER people's charts

You'll write small charts for your own apps — but the bigger win is installing complex third-party software (Prometheus stack, ingress-nginx, Postgres) as one command instead of 40 hand-written manifests. That's exactly what M8's lab does with kube-prometheus-stack.

Your real project: billfree deploys via its own Helm chart — one chart, env-specific values files. Deep dive with real chart structure → ch28.

Apna Helm chart likhna — from scratch (`_helpers.tpl` + named templates)¶

Kyun zaroori hai? Third-party chart consume karna alag cheez hai — 2-3 saal ke engineer se expect kiya jaata hai ki woh company ka apna internal chart likhe. Billfree exactly yahi karti hai: ek chart, multiple values-*.yaml files.

Step 1 — `helm create mychart` — skeleton milta hai¶

helm create mychart   # ek poori working chart scaffold karta hai — explore karo!

Generated structure:

mychart/
├── Chart.yaml              # package identity: name, version, appVersion
├── values.yaml             # default answers (every blank ka fallback)
├── values.schema.json      # (by default nahi hota — aap banate ho) typo validator
└── templates/
    ├── _helpers.tpl        # ← DRY functions — koi Kubernetes resource nahi, sirf named templates
    ├── deployment.yaml
    ├── service.yaml
    ├── serviceaccount.yaml
    ├── hpa.yaml
    ├── ingress.yaml
    └── NOTES.txt           # text printed after helm install — onboarding hint

_ (underscore) se shuru hone wali file ko Helm Kubernetes me render nahi karta — yeh sirf internal helper file hai.

Step 2 — WHY `_helpers.tpl`? DRY labels¶

Ek realistic chart me yahi labels baar-baar copy-paste hoti hain:

File	Kahan
`deployment.yaml`	`metadata.labels`, `spec.selector.matchLabels`, `spec.template.metadata.labels`
`service.yaml`	`metadata.labels`, `spec.selector`
`serviceaccount.yaml`	`metadata.labels`

Teen jagah, ek typo — poora chart toot jaata hai. _helpers.tpl me ek baar define, har jagah include karo:

{{/*  mychart/templates/_helpers.tpl  */}}

{{/*
Chart name — 63 chars tak trim (Kubernetes label value limit).
*/}}
{{- define "mychart.name" -}}
{{- .Chart.Name | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Common labels — metadata.labels me use hota hai.
Includes chart version, appVersion, managed-by — upgrade pe change ho sakte hain (OK here).
*/}}
{{- define "mychart.labels" -}}
helm.sh/chart: {{ printf "%s-%s" .Chart.Name .Chart.Version | trunc 63 | trimSuffix "-" }}
{{ include "mychart.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{/*
Selector labels — spec.selector.matchLabels + spec.template.metadata.labels me use hota hai.
SIRF stable identity labels — version NAHI (see warning below).
*/}}
{{- define "mychart.selectorLabels" -}}
app.kubernetes.io/name: {{ include "mychart.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

Step 3 — Templates me `include` karna¶

# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "mychart.name" . }}-{{ .Release.Name }}
  labels:
    {{- include "mychart.labels" . | nindent 4 }}         # full labels block, 4-space indent
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      {{- include "mychart.selectorLabels" . | nindent 6 }} # SIRF selector labels — not full labels
  template:
    metadata:
      labels:
        {{- include "mychart.selectorLabels" . | nindent 8 }}
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          ports:
            - containerPort: {{ .Values.service.port }}

# templates/service.yaml — same selectorLabels template, zero copy-paste
apiVersion: v1
kind: Service
metadata:
  name: {{ include "mychart.name" . }}
  labels:
    {{- include "mychart.labels" . | nindent 4 }}
spec:
  selector:
    {{- include "mychart.selectorLabels" . | nindent 4 }}
  ports:
    - port: {{ .Values.service.port }}
      targetPort: {{ .Values.service.port }}

Production gotcha — selectorLabels me version label kabhi mat daalo

spec.selector.matchLabels Kubernetes me immutable hai — ek baar Deployment bana, selector kabhi nahi badal sakta (bina delete + recreate ke). Agar selectorLabels me app.kubernetes.io/version: {{ .Chart.AppVersion }} daal do, to har helm upgrade pe version badlega → selector change hoga → Kubernetes conflict error dega, upgrade fail hoga. Issi liye standard _helpers.tpl me selectorLabels = sirf name + instance (stable, kabhi nahi badalta). version sirf labels (non-selector) block me jaata hai — wahan OK hai. Rule: Selector = app identity (stable forever). Labels = full metadata (can change).

include vs template — ek line

{{ include "mychart.labels" . | nindent 4 }} — pipe chain kaam karta hai. ✅ {{ template "mychart.labels" . | nindent 4 }} — kaam nahi karta (template action pipeline support nahi karta). Production me hamesha include use karo.

Step 4 — `values.schema.json` — typo fast-fail karo¶

Bina schema ke helm install ./mychart --set replicaCount=abc silently broken Deployment banata hai. Schema file hone se install hi fail ho jaata hai — CI me fail fast, prod me surprise nahi:

{
  "$schema": "https://json-schema.org/draft-07/schema#",
  "properties": {
    "replicaCount": {
      "description": "Number of pod replicas",
      "type": "integer",
      "minimum": 1
    },
    "image": {
      "type": "object",
      "properties": {
        "repository": { "type": "string" },
        "tag":        { "type": "string" }
      },
      "required": ["repository", "tag"]
    }
  },
  "required": ["replicaCount", "image"]
}

helm install demo ./mychart --set replicaCount=abc
# Error: values don't meet the specifications of the schema(s) in the following chart(s):
#   mychart: replicaCount: Invalid type. Expected: integer, given: string

CI me fast-fail >> prod me mysterious crashloop.

Step 5 — End-to-end: likhne ke baad kya karo¶

# 1. Render karke aankhon se dekho — cluster touch nahi hota
helm template myrelease ./mychart

# 2. Lint — common mistakes + best practices (missing labels, deprecated APIs, etc.)
helm lint ./mychart

# 3. Dry-run with debug — Kubernetes API validate karta hai, koi resource nahi banta
helm install myrelease ./mychart --dry-run --debug

# 4. Asli install (ya CI me: upgrade --install — install if absent, upgrade if present)
helm install myrelease ./mychart -f values-prod.yaml

helm template → render dekho. helm lint → chart errors pakdo. --dry-run --debug → cluster-level validation. Teeno karo, tab apply karo.

Try it — 10 min

helm create myapp && cd myapp
# templates/_helpers.tpl kholo — "selectorLabels" define dhundo, pado
helm template myrelease .
helm lint .
helm install myrelease . --dry-run --debug | grep -A 8 "kind: Deployment"
# Deployment ke selector.matchLabels me version label hai? (nahi hona chahiye — kyun?)

Bonus: values.yaml me replicaCount: "two" karo (string, not int), upar wala values.schema.json banao, phir helm lint . chalao — schema validation ka error dekhna. Schema ki value tab samajh aati hai jab woh kaam karta hai.

🇮🇳 Hinglish takeaway: helm create ready skeleton deta hai, _helpers.tpl usme DRY labels ka function library hai. Selector chota rakho aur stable — version wahan mat daalo warna upgrade fail karega. include use karo, template nahi. helm template → lint → dry-run — teen steps, tab cluster touch karo.

Kustomize in 10 minutes¶

The two ideas¶

Idea	What it is	Analogy
base/	Plain, complete, runnable YAML — no placeholders at all	Original letter
overlay/	Per-environment folder listing ONLY the differences (patches)	Sticky notes on top

What it looks like¶

app/
├── base/
│   ├── deployment.yaml       # 100% normal YAML — kubectl apply would work on it
│   ├── service.yaml
│   └── kustomization.yaml    # "ye files मेरी base hैं"
└── overlays/
    ├── dev/
    │   └── kustomization.yaml     # "base lo, replicas=1 kar do"
    └── prod/
        ├── kustomization.yaml     # "base lo, ye patch lagao"
        └── replica-patch.yaml     # sirf jo badla: replicas: 5

# overlays/prod/kustomization.yaml
resources:
  - ../../base            # start from the base
patches:
  - path: replica-patch.yaml
images:
  - name: myapp
    newTag: "2.0.1"       # image tag bhi yahin override hota hai

# overlays/prod/replica-patch.yaml — ONLY the diff, nothing else
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 5

The commands that matter¶

kubectl kustomize overlays/prod       # render and SEE the final YAML (like helm template)
kubectl apply -k overlays/prod        # render + apply (-k is built into kubectl!)
cd overlays/prod && kustomize edit set image myapp=myapp:abc1234
                                      # ← THE CI command: updates the image tag in
                                      #   kustomization.yaml — this is what ch19's pipeline runs

No templates = the whole point

The base YAML is valid, runnable Kubernetes YAML — readable by anyone, lintable by any tool, no {{ }} syntax to learn. That simplicity is why Google built it into kubectl itself.

Your real project: VANTA-Boutique uses Kustomize — one base, overlays per environment. Hands-on with the real repo → Lab B · VANTA.

🎛️ 🎬 Dekho: ek YAML, dev se prod tak — bina copy-pastePrinter (template+values) vs sticky-note (base+overlay), decision tree, Argo — animated. ▶ Kholo

Helm vs Kustomize — the decision, once and for all¶

flowchart TD
    Q1{"Third-party software?<br/>(Prometheus, Postgres, ingress...)"}
    Q2{"Need versioned releases +<br/>one-command rollback +<br/>distributing to other teams?"}
    Q3{"Just environment differences<br/>on YOUR OWN app?"}
    H["🎩 HELM<br/>charts exist for everything —<br/>never hand-write Prometheus YAML"]:::helm
    H2["🎩 HELM<br/>release history = rollback +<br/>chart = shareable package"]:::helm
    K["📄 KUSTOMIZE<br/>plain YAML + tiny overlays,<br/>zero template syntax"]:::kust

    Q1 -->|yes| H
    Q1 -->|no| Q2
    Q2 -->|yes| H2
    Q2 -->|no| Q3
    Q3 -->|yes| K

    classDef helm fill:#f3e5f5,stroke:#8e24aa,color:#4a148c;
    classDef kust fill:#e0f2f1,stroke:#00897b,color:#004d40;

Dimension	Helm	Kustomize
Strategy	Templates + values (fill blanks)	Base + overlays (patch diffs)
Learning curve	Template syntax to learn	Zero new syntax
Base YAML readable alone?	No — full of `{{ }}`	Yes — plain valid YAML
Rollback	`helm rollback` (built-in history)	Via Git only
Third-party software	✅ its killer feature (chart repos)	✗ not its job
Built into kubectl	No (separate binary)	✅ `kubectl apply -k`
With Argo CD (M7)	Natively rendered	Natively rendered
Your projects	billfree	VANTA-Boutique

Interview one-liner: "Kustomize for our own app's environment variants — plain YAML, no templating. Helm where we need packaging: third-party charts or versioned releases with rollback. They're not rivals — most shops run both, and Argo CD renders either natively."

🇮🇳 Hinglish intuition: Helm = card printer hai (naya package chhapta hai, purane sab records me). Kustomize = sticky note hai (original safe, upar note). Printer tab jab poora package bantna ho; note tab jab bas 2-3 cheez badalni ho.

Hands-on lab (20 min, kind cluster)¶

✅ Prove it — bash labs/check-m75-helm-kustomize.sh

Lab ho gaya? Tick mat lagao — machine se verify karo (live release · revision>1 rollback · overlay renders). ❌ pe exact fix-hint. → The Doer's Path

Lab 1 — Helm: one chart, two environments¶

helm create demo                  # scaffolds a full working chart — explore it!
helm template web ./demo | head -40    # SEE what the blanks render to

# dev install (defaults: 1 replica)
helm install web-dev ./demo

# "prod" install — same chart, different answers, no YAML edited:
helm install web-prod ./demo --set replicaCount=3
kubectl get deploy                # web-dev: 1/1 · web-prod: 3/3  ← same chart, two shapes

# rollback drill:
helm upgrade web-prod ./demo --set image.tag=broken
helm history web-prod             # revision 2 = the mistake
helm rollback web-prod 1          # one command, back to good
helm uninstall web-dev web-prod   # cleanup

Lab 2 — Kustomize: one base, two overlays¶

mkdir -p app/base app/overlays/dev app/overlays/prod

# base = the plain YAML you already know how to write (M4)
cat > app/base/deployment.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata: {name: web, labels: {app: web}}
spec:
  replicas: 1
  selector: {matchLabels: {app: web}}
  template:
    metadata: {labels: {app: web}}
    spec:
      containers: [{name: nginx, image: nginx:1.27}]
EOF
cat > app/base/kustomization.yaml <<'EOF'
resources: [deployment.yaml]
EOF

# dev overlay = base as-is
cat > app/overlays/dev/kustomization.yaml <<'EOF'
resources: [../../base]
EOF

# prod overlay = base + 5 replicas + newer image
cat > app/overlays/prod/kustomization.yaml <<'EOF'
resources: [../../base]
patches:
  - patch: |-
      apiVersion: apps/v1
      kind: Deployment
      metadata: {name: web}
      spec: {replicas: 5}
images:
  - name: nginx
    newTag: "1.27.1"
EOF

kubectl kustomize app/overlays/prod   # LOOK: replicas 5 + new tag, base untouched
kubectl apply -k app/overlays/dev     # deploy dev
kubectl get deploy web                # 1/1 — the base as-is

✅ Sahi hua to: Lab 1 me web-dev 1 replica aur web-prod 3 replicas — ek hi chart se; helm rollback ke baad broken tag gayab. Lab 2 me kubectl kustomize prod render me replicas: 5 + nginx:1.27.1 dikhata hai jabki base file me ab bhi replicas: 1 + 1.27 hi likha hai.

Memory shortcuts¶

Helm = shaadi-card printer 🎴 — template + answers(values) → package(chart) → chhapa hua batch(release).
Kustomize = sticky notes 📝 — original(base) kabhi mat chhedo; note(overlay) chipkao.
helm template / kubectl kustomize = pehle dekho, phir lagao — dono ka dry-run.
Rollback: Helm ke paas apni history hai; Kustomize me rollback = git revert (GitOps way, M7).
billfree = Helm · VANTA = Kustomize — tumhare dono projects, dono pattern. Coincidence nahi — industry aisi hi hai.

Self-check quiz¶

Security patch ek Deployment spec me lagana hai jo 3 environments me chalta hai. Copy-paste setup me kya risk hai, aur base+variation me kya guarantee?
helm template aur kubectl kustomize common kya karte hain, aur ye habit production me kyon zaroori hai?
CI pipeline image tag update karti hai. Helm world me kya badlta hai, Kustomize world me kaun si command chalti hai?
Prometheus stack deploy karna hai — Helm ya Kustomize? Kyun ek line me.
helm rollback hai to bhi GitOps team git revert kyun prefer karti hai? (M7 yaad karo)

Jawab dekho

Copy-paste: teen jagah edit — ek bhool gaye to prod me hole. Base+variation: ek edit, teeno environments me guaranteed same fix.
Dono final YAML render karke dikhaते hain bina apply kiye — "preview before apply." Blind apply production me surprise deta hai; render diff review = deploy review.
Helm: values me image.tag badalta hai (ya --set image.tag=$SHA). Kustomize: kustomize edit set image app=app:$SHA — yahi command ch19 ki CI chalati hai.
Helm — community chart maujood hai (kube-prometheus-stack); 40+ manifests haath se likhna bewakoofi hai.
helm rollback cluster ko peeche le jaata hai par Git aage hi rehta hai — agla Argo sync rollback ko undo kar dega (split brain). git revert Git ko hi theek karta hai, Argo cluster ko follow karata hai. Git = source of truth.

← M7 · GitOps Part II · The Connected System →