The Connected System¶
How all seven tools hand off to each other, end to end¶
Core question: You've learned the tools one at a time. But production is not seven islands โ it's one machine with seven moving parts. Where exactly does each tool stop, and the next one begin?
โฑ๏ธ Time: ~40 min padho ยท ๐๏ธ Level: Intermediate (synthesis) ยท ๐ Pehle chahiye: M0โM7, M7.5
Is module ke baad tum kar paoge: - 2 loops, 8 bridges, 5 threads whiteboard pe draw karo โ bina notes ke - Kisi bhi production failure ka diagnosis shuru karo: which bridge broke? - Ek
git pushse user ke browser tak ka poora path ek sentence mein sunaao โ 30-minute interview answer ki tarahโฉ๏ธ Recall gate โ shuru karne se pehle¶
Pichhle modules se 3 sawaal. Pehle memory se jawab do, phir kholo. (Yeh retrieve karna hi lifetime yaad rakhta hai โ dobara padhna nahi.)
- (M7) Argo CD "pull model" hai โ push model (CI direct deploy) ke mukable mein iska ek concrete security fayda kya hai?
- (M6) CI pipeline ne image ECR pe push kar di. Cluster mein naya version pohunchane ke liye CI seedha
kubectl applykarta hai โ sahi ya galat? Agar galat, toh kya karta hai?- (M4) K8s reconciliation loop kya compare karta hai, aur agar koi pod manually delete kar do, toh kya hota hai?
Jawab
- Pull mein Argo cluster ke andar se Git read karta hai โ cluster credentials kabhi bahar nahi jaate; CI ko kubeconfig ki zaroorat nahi. 2. Galat โ CI sirf k8s manifest mein image tag update karta hai aur Git commit karta hai (Bridge 5); Argo baad mein Git se pull karta hai. 3. Desired state (spec replicas:3) vs current state (live pods:2) โ controller loop missing pod wapas banata hai desired poora karne ke liye.
This is the chapter that turns a pile of tools into a mental model. Read it after M0โM7. Everything here is synthesis โ no new tools, just the joints between the ones you know. Master this and a whiteboard interview becomes a story you can tell for 30 minutes without pausing.
The 60-second version¶
The whole stack is two reconciliation loops sharing one Git repo. The outer loop (Terraform โ Ansible) builds the place things run โ rarely, carefully; its servers are Pets. The inner loop (Git โ CI โ Registry โ GitOps โ Kubernetes โ Users) ships the app โ constantly, automatically; its pods are Cattle. Between every pair of tools is a handoff โ a specific output one tool produces that the next consumes. There are 8 of these bridges. And under all of it run 5 golden threads โ ideas (reconciliation, state-outside, preview-before-apply, push-vs-pull, idempotency) that repeat in every single tool.
๐ง Memory hook: 2 loops, 8 bridges, 5 threads. If you can draw those three things, you understand DevOps.
1. The two loops (the map)¶
flowchart LR
Git["GIT<br/>source of truth"]:::shared
TF["Terraform<br/>VPC ยท EC2 ยท RDS ยท ECR"]:::infra
Ans["Ansible<br/>kubeadm ยท Calico"]:::infra
K8s["Kubernetes Cluster<br/>where loops meet"]:::run
CI["CI / Actions<br/>test + build"]:::ci
ECR["Registry / ECR<br/>immutable image"]:::ci
Argo["Argo CD<br/>pull-based GitOps"]:::ci
Users(["Users"]):::run
Git -->|"outer loop"| TF
TF -->|"B1 ยท server IPs"| Ans
Ans -->|"B2 ยท live cluster"| K8s
Git -->|"inner loop"| CI
CI -->|"B3 ยท image push"| ECR
CI -->|"B5 ยท tag commit"| Git
ECR -. "B3 ยท kubelet pull" .-> K8s
Git -. "B6 ยท watches" .-> Argo
Argo -->|"B6 ยท apply"| K8s
K8s --> Users
classDef shared fill:#fff9c4,stroke:#f9a825,color:#4a3800;
classDef infra fill:#fce4ec,stroke:#d81b60,color:#880e4f;
classDef ci fill:#e3f2fd,stroke:#1976d2,color:#0d47a1;
classDef run fill:#e0f2f1,stroke:#00897b,color:#004d40;
Outer loop (Pets, runs rarely) and inner loop (Cattle, every push) both read from Git and converge at the Kubernetes cluster.
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ GIT โ single source of truth โ
โ (infra code + app code + K8s manifests) โ
โโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโ
โ โ
โโโโโโโ OUTER LOOP โโโโโโโโโโผโโโโโโโ โโโโโโโโโโโโโผโโโโโโโ INNER LOOP โโโโโโโโโโโ
โ INFRASTRUCTURE ("build the โ โ DELIVERY ("ship the app") โ
โ house") โ runs RARELY, Pets โ โ runs EVERY PUSH, Cattle โ
โ โ โ โ
โ Terraform โโถ Cloud (VPC, EC2, โ โ Code โโถ CI (test+build) โโถ image โโถ โ
โ RDS, ECR, S3/DynamoDB) โ โ Registry (ECR) โโถ manifest-update in โ
โ โ โ โ Git โโถ Argo CD (pull) โโถ Kubernetes โโถ โ
โ โผ โ โ Service โโถ Ingress โโถ LB โโถ DNS โโถ โ
โ Ansible (configure hosts, โ โ Users โ
โ kubeadm, Calico) โโถ live cluster โ โ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
The two loops meet in exactly one place: the Kubernetes cluster. The outer loop produces it; the inner loop deploys into it. Git is the shared brain both loops read from.
๐ฎ๐ณ Hinglish intuition: Outer loop = neev aur ghar banao (ek baar, mazbooti se). Inner loop = rozana khana banao aur serve karo (baar-baar, machine se). Dono ki common cheez ek hi: recipe book (Git).

Figure 1: the two loops โ outer (infra/Pets) and inner (delivery/Cattle) โ meeting at the Kubernetes cluster, Git as the shared source of truth.

Figure 2: the same system unrolled left-to-right โ every handoff (the 8 bridges below) shown as an arrow from one tool to the next.
Full-resolution renders (and extra views) live in the repo root: devops_two_loops_mental_model.png, two_loops_infra_vs_app.png, devops_full_pipeline_handoffs.png, full_production_flow_end_to_end.png, complete_production_pipeline_flow.png, kubernetes_controller_chain.png.
2. The 8 Bridges¶
A bridge is a handoff: the concrete artifact tool A produces that tool B consumes. Learn the artifact and the failure mode when the bridge breaks, and you can debug any seam.
[Terraform]โโ โโถ[Ansible]โโกโโถ[Kubernetes cluster] โ outer loop bridges
โ โฒ
โ(builds ECR, RDS) โโข (image pulled from ECR into a Pod)
โผ โ
[Docker image]โโขโโถ[Registry/ECR]โโโโโโโโโโโโ
[git push]โโฃโโถ[CI/Actions]โโคโโถ[Git manifest]โโฅโโถ[Argo CD]โโถ[Kubernetes]
[Pod]โโฆโโถ[Service]โโถ[Ingress]โโถ[User] [Pod]โโงโโถ[RDS]
| # | Bridge | The artifact that crosses | Failure mode when it breaks |
|---|---|---|---|
| 1 | Terraform โ Ansible | terraform output server public IPs โ written into Ansible's inventory.ini |
Wrong/empty IP โ Ansible UNREACHABLE |
| 2 | Ansible โ Kubernetes | 3 playbooks (common โ kubeadm init on master โ kubeadm join on workers) produce a live cluster |
Missing containerd/swap-on โ kubeadm fails; wrong join token โ worker never joins |
| 3 | Docker โ Pod | image pushed to ECR, then pulled by kubelet onto a node | Bad tag / no ECR auth โ ImagePullBackOff |
| 4 | Push โ CI | a git push to main matches on: push: โ runs the workflow |
Wrong branch filter โ pipeline never triggers |
| 5 โญ | CI โ Git manifest | CI updates the image tag inside the K8s manifest (via sed) and commits โ it does not apply to the cluster |
Forgot to update the manifest โ cluster keeps the old image forever |
| 6 | Git โ Argo โ Kubernetes | Argo pulls the changed manifest and applies it โ rolling update | Argo OutOfSync never syncs โ new version never lands |
| 7 | Pod โ Service โ User | Service (stable IP) fronts ready pods via EndpointSlice; NodePort/Ingress exposes it | No ready pods in the slice โ Connection refused / 503 |
| 8 | Pod โ RDS | app connects psycopg2 โ rds-endpoint:5432 using an env-var host + a secret password |
Wrong SG / password โ connection timeout or auth error |
Bridge 5 is the one interviewers probe. Say it precisely: "CI never touches the cluster. It only changes the desired state in Git โ updates the image tag and commits. Argo CD, running inside the cluster, notices Git changed and pulls it. That separation is why the pipeline needs no cluster credentials sitting in GitHub."
๐ฎ๐ณ Hinglish intuition (Bridge 5): CI menu update karta hai (Git me naya tag likhta), khud khana serve nahi karta. Serve karna Argo ka kaam hai. Isliye CI ke paas cluster ki chaabi rakhne ki zaroorat nahi.
3. The 5 Golden Threads¶
If the 8 bridges are the joints, the 5 threads are the DNA โ the same idea wearing a different costume in each tool. Spot them and 9 tools collapse into 1 idea.
Thread 1 โ Reconciliation¶
A control loop constantly compares desired state to current state and drives current โ desired. It never stops.
| Tool | Desired state | Who reconciles |
|---|---|---|
| Terraform | .tf files + tfstate |
you, when you run apply |
| Ansible | playbook state=present |
you, when you run the playbook |
| Kubernetes | Deployment replicas: 3 |
the controller loop, continuously |
| Argo CD | the manifest in Git | Argo, continuously (~3 min) |
The leap: Terraform/Ansible reconcile when you ask; Kubernetes/Argo reconcile forever, on their own. That's why a deleted pod comes back and a hand-edited object gets reverted. Learn reconciliation once in M4, and Argo's selfHeal is free.
๐ฎ๐ณ Hinglish intuition: Chowkidar jo 24/7 ginta hai โ "3 hone chahiye, 2 hain, ek aur banao." Yehi loop har jagah hai.
Thread 2 โ State outside, compute disposable¶
Keep durable state (database,
tfstate) outside the throwaway compute. Then servers and pods become cattle โ kill and replace freely.
tfstateโ S3 (not your laptop) โ any machine can run Terraform.- App is stateless โ pod dies โ identical replacement, no data lost.
- DB is stateful โ usually lives in RDS outside the cluster, backed up โ never a casual pod (i.e. never a bare Deployment with an
emptyDir). Running it in-cluster is legitimate, but only deliberately: a StatefulSet withvolumeClaimTemplates, tested backups, and someone owning failover. See the explicit trade-off in 28 โ Helm in the real world, where billfree runs a self-managed Postgres StatefulSet on purpose. Managed = less control, far less operational burden; self-managed = full control, and the pager is yours. - Because compute is disposable, you can run it on Spot instances (M5) and save 70%.
๐ฎ๐ณ Hinglish intuition: State bahar nikaalo โ server/pod disposable ban jaata hai. Yehi cattle-not-pets ka asli mechanism hai.
Thread 3 โ Preview before apply¶
Never change reality blind. Every tool has a dry-run.
| Tool | Preview command | It shows |
|---|---|---|
| Terraform | terraform plan |
the diff before it touches AWS |
| Ansible | ansible-playbook --check |
what would change |
| Kubernetes | kubectl apply --dry-run=server |
validation without applying |
| CI/CD | the test gate | "does it even build & pass?" before shipping |
๐ฎ๐ณ Hinglish intuition: Bill dekho, phir payment karo.
plan= bill,apply= payment.
Thread 4 โ Push vs Pull¶
Who initiates the change โ the doer reaching in, or the target reaching out?
| Model | Tools | Security consequence |
|---|---|---|
| Push | Ansible, GitHub Actions | The initiator holds the target's credentials (SSH keys, cluster creds) |
| Pull | Argo CD | The agent lives inside the target and pulls โ no external creds leave the cluster |
| The middle | Git | Neither pushes nor pulls; it just holds the truth both sides agree on |
This is why GitOps is considered more secure than CI-driven kubectl apply: with pull, GitHub never needs your cluster's admin credentials.
Thread 5 โ Idempotent¶
Run it again, nothing new happens. The operation is
SET, not+=.
terraform apply twice โ no duplicate VPC. Ansible playbook twice โ second run is all ok, zero changed. kubectl apply twice โ same object. This is what makes automation safe to retry โ the single most important property for a tool a machine runs unattended.
๐ฎ๐ณ Hinglish intuition: Switch (idempotent: "ON" dabao 10 baar, bulb ON hi rahega) vs Counter (
+=: har dabane pe badhta). DevOps tools switch jaise hain.
4. The full end-to-end walkthrough¶
Now watch a single feature travel the entire system, crossing every bridge in order.
Day 0 โ Setup (outer loop, done once)¶
1. terraform apply โ VPC, 3ร EC2, RDS, ECR, S3+DynamoDB lock created [state โ S3, Thread 2]
2. terraform output โ 3 server IPs [Bridge 1]
3. edit inventory.ini โ paste the IPs [Bridge 1]
4. ansible-playbook 1-common.yml โ containerd, swapoff, sysctl on all 3 [Thread 5: idempotent]
5. ansible-playbook 2-master.yml โ kubeadm init + Calico CNI [Bridge 2]
6. ansible-playbook 3-workers.yml โ kubeadm join ร 2 [Bridge 2]
โถ Result: a live Kubernetes cluster. The two loops now share this cluster.
flowchart LR
Push["git push<br/>to main"]:::ci
Test["CI ยท pytest<br/>test gate"]:::ci
Build["CI ยท docker build<br/>push to ECR"]:::ci
ManifestUp["CI updates<br/>image tag in Git"]:::ci
ArgoSync["Argo CD<br/>OutOfSync then apply"]:::ci
RollingUp["K8s rolling update<br/>new ReplicaSet"]:::run
Svc["Service<br/>EndpointSlice"]:::run
RDS[("RDS<br/>state outside")]:::run
User(["User<br/>200 OK"]):::run
Push -->|"B4 triggers"| Test
Test -->|"pass"| Build
Build -->|"B3 image in ECR"| ManifestUp
ManifestUp -->|"B5 commit"| ArgoSync
ArgoSync -->|"B6 apply"| RollingUp
RollingUp -->|"B7 ready pods"| Svc
Svc --> User
RollingUp -->|"B8 connect"| RDS
classDef ci fill:#e3f2fd,stroke:#1976d2,color:#0d47a1;
classDef run fill:#e0f2f1,stroke:#00897b,color:#004d40;
Every git push crosses Bridges 4 โ 5 โ 6 โ 7 (and Bridge 8 for the database) before reaching the user.
Day 1+ โ Delivery (inner loop, every push)¶
7. git push origin main โ matches on: push: [main] [Bridge 4]
8. CI: pytest โ test gate; fail = nothing ships [Thread 3]
9. CI: docker build + push to ECR โ image tagged with ${{ github.sha }} [Bridge 3, immutable tag]
10. CI: sed image tag in k8s/deploy.yaml โ git commit + push [Bridge 5 โญ: change Git, NOT cluster]
11. Argo CD notices Git changed โ 3-way diff โ OutOfSync โ apply [Bridge 6, Thread 1]
12. Kubernetes rolling update โ new ReplicaSet, readiness-gated [Thread 1]
13. Service EndpointSlice updates โ traffic shifts to new pods [Bridge 7]
14. Pod โ RDS:5432 โ serves the request [Bridge 8]
15. User hits Ingress โ Service โ Pod โ 200 OK
When something goes wrong¶
Bug in prod? โ git revert <bad commit> โ Argo pulls the old manifest โ rollback. [Thread 1 + Git = time machine]
Someone kubectl-edited a live pod? โ Argo selfHeal reverts it to match Git. [Thread 1 + Thread 4]
tfstate deleted? โ Terraform thinks NOTHING exists โ would try to rebuild everything. [Thread 2: guard your state]
๐ฎ๐ณ Hinglish intuition (rollback): Time machine โฉ๏ธ โ galti hui to Git me peeche jao, Argo cluster ko wapas us waqt pe le aayega. Rollback = sirf Git history.
5. The connection web (one glance)¶
flowchart TD
TF["Terraform<br/>builds VPC ยท EC2 ยท RDS ยท ECR ยท S3"]:::infra
Ans["Ansible<br/>kubeadm ยท cluster bootstrap"]:::infra
Git["GIT<br/>source of truth"]:::shared
CI["CI / Actions"]:::ci
ECR["Registry / ECR"]:::ci
Argo["Argo CD<br/>pull-based"]:::ci
K8s["K8s Cluster<br/>Deployment ยท Service ยท Ingress"]:::run
RDS[("RDS<br/>state outside cluster")]:::run
Users(["Users"]):::run
TF -->|"B1 ยท IPs"| Ans
TF -->|"creates"| ECR
TF -->|"creates"| RDS
Ans -->|"B2 ยท live cluster"| K8s
Git -->|"B4 ยท triggers"| CI
CI -->|"B3 ยท image push"| ECR
CI -->|"B5 ยท tag update"| Git
ECR -. "B3 ยท kubelet pull" .-> K8s
Git -. "B6 ยท watches" .-> Argo
Argo -->|"B6 ยท apply"| K8s
K8s -->|"B7"| Users
K8s -->|"B8 ยท connect"| RDS
classDef shared fill:#fff9c4,stroke:#f9a825,color:#4a3800;
classDef infra fill:#fce4ec,stroke:#d81b60,color:#880e4f;
classDef ci fill:#e3f2fd,stroke:#1976d2,color:#0d47a1;
classDef run fill:#e0f2f1,stroke:#00897b,color:#004d40;
All eight bridges in one view โ Terraform builds the foundation, Git is the hub both loops read from, all paths converge at the Kubernetes cluster.
โโโโโโโโโโโโโ TERRAFORM โโโโโโโโโโโโโ
โ builds: VPC, EC2, RDS, ECR, S3 โ
โโโโโโโโโฌโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโ
IPs (Bridge1)โ โcreates ECR & RDS
โผ โ
ANSIBLE โ
(kubeadm โ cluster, Bridge2) โ
โ โ
โโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโ
โ KUBERNETES CLUSTER โ
โ DeploymentโReplicaSetโPod โข ServiceโIngress โ
โ โฒ pulls image (Bridge3) โ serves users โ
โโโโโโโโโโโโโโโโโโโโโโโโโฒโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโ
โ โBridge6 โ Bridge7โ USERS
ECR โ ARGO CD (pull) โ
โ โฒ โ Bridge8
โโโโโโโโโโดโโโโโโโ โBridge5 โผ
โ DOCKER image โโโโ CI/ACTIONS โโ GIT โโโถ RDS (state, outside)
โ (Bridge4: git push triggers CI) โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Threads woven through ALL boxes: โ reconcile โก state-outside โข preview โฃ push/pull โค idempotent
6. Who owns what, in a real org¶
Systems thinking includes people. On a real team these boxes have different owners โ knowing this is what separates a junior from someone who's shipped in production.
| Layer | Typical owner | What they're paged for |
|---|---|---|
| Terraform / cloud infra | Platform / Infra / Cloud team | VPC, IAM, account limits, cost |
| Ansible / host config | Platform / SRE | drift, patching, cluster bootstrap |
| Docker image / app code | Product / app developers | the app builds and runs |
| CI/CD pipeline | Platform provides, devs consume | broken builds, flaky tests |
| Kubernetes cluster | Platform / SRE | node health, capacity, upgrades |
| GitOps / Argo | Platform / SRE | sync failures, prod drift |
| Observability & on-call | SRE + the owning dev team | SLO burn, incidents (see M8) |
| The database (RDS) | DBA / data platform | backups, failover, slow queries |
Senior insight: the boundaries above are also blast-radius boundaries. When you design a change, ask "which of these owners do I need to tell, and what breaks if this box fails?" That question is 80% of an architecture review.
7. Who does what โ the tool recall drill¶
Section 6 was about people. This one is about tools: which tool owns which job. Interviewers probe this constantly ("who deploys to the cluster?", "who runs the container?") โ so drill it until the answer is instant. Cover the right column and quiz yourself.
The layer stack (memorize this ONE picture):
App โ Pods / Containers โ KUBERNETES (schedules, restarts, scales)
Node OS: containerd, kubeadm โ ANSIBLE (installs & configures the node)
The server: VM, VPC, subnet, disk โ TERRAFORM (creates the infrastructure)
๐ฎ๐ณ Hinglish intuition: "Terraform building banata, Ansible kitchen set karta (stove/utensils = containerd/kubeadm), Kubernetes chef hai jo dishes (pods) pakata." Teen tools, teen layers, isi order me. Ansible cook nahi karta; K8s plumbing nahi bichata.
Task โ owner (the master table):
| The task | Who does it | Layer |
|---|---|---|
| Create the VM / VPC / disk | Terraform | infra |
| Install runtime + kubeadm, disable swap, kernel tuning | Ansible | node OS |
| Form the cluster (init/join) | kubeadm (run by Ansible) | node OS |
| Build the container image | Docker (on the CI runner) | build |
| Store the image | Registry (ECR / GHCR / Hub) | build |
| Actually run a container | containerd (under kubelet) | node |
| Decide which node a pod runs on | kube-scheduler | control plane |
| Keep N replicas alive (self-heal) | Deployment โ ReplicaSet | cluster |
| Route traffic to the right pods | Service (+ kube-proxy) / Ingress | cluster |
| Persist data beyond a pod | PVC โ PV (โ EBS) | storage |
| Hold config / secrets | ConfigMap / Secret | cluster |
| Test โ build โ scan the code | CI (Jenkins / GitHub Actions) | pipeline |
| Deploy to the cluster | ArgoCD (GitOps) โ not CI! | delivery |
| Observe (metrics/logs/alerts) | Prometheus / Grafana / Loki | observability |
โ ๏ธ The trap most people fall into: thinking CI deploys. It doesn't โ CI writes to Git; ArgoCD reads from Git and deploys. That separation (Bridge 5) is the whole point of GitOps.
Where does each tool RUN? (another recall-killer)
| Laptop / CI runner (clients) | Master node | Worker nodes |
|---|---|---|
terraform (calls cloud API) |
api-server + etcd | kubelet |
ansible (SSH push) |
scheduler, controllers | kube-proxy |
kubectl (calls API) |
ArgoCD (pods) | containerd โ your pods run here |
docker build, git push |
Prometheus / Grafana (pods) |
Golden rule: clients (Terraform, Ansible, kubectl, docker-build) run from your laptop/CI and command remote things; servers (containerd, kubelet, ArgoCD) run on the cluster. ArgoCD lives inside the cluster โ that's what makes GitOps pull-based.
Rapid-fire (cover the answers, drill until instant):
Who does X? โ jawab dekho
- Creates the EC2 instance? โ Terraform
- Installs containerd + kubeadm? โ Ansible
- Forms the cluster? โ kubeadm
- Builds the Docker image? โ CI (Docker on the runner)
- Stores the image? โ Registry
- Runs the container on a node? โ containerd (under kubelet)
- Decides which node? โ scheduler
- Recreates a dead pod? โ ReplicaSet (via the Deployment)
- Load-balances to pods? โ Service
- Routes external HTTP by path? โ Ingress
- Gives a pod persistent storage? โ PVC โ PV (EBS)
- Deploys the new image to the cluster? โ ArgoCD (from Git) โ not CI
- Watches Git for changes? โ ArgoCD
- Collects metrics? โ Prometheus ยท Shows dashboards? โ Grafana
๐ The actual commands for each tool live in Command Cheat-Sheets & Labs โ this drill is who, that chapter is how.
Summary¶
- The stack is 2 loops (infra/outer/Pets + delivery/inner/Cattle) sharing 1 Git repo; they meet at the Kubernetes cluster.
- 8 bridges are the handoffs; each has a concrete artifact and a signature failure mode. Bridge 5 (CI changes Git, not the cluster) is the one to nail.
- 5 golden threads โ reconciliation, state-outside, preview-before-apply, push-vs-pull, idempotent โ recur in every tool. Learn each once, reuse everywhere.
- A feature travels:
push โ test โ build โ ECR โ manifest-update โ Argo โ rolling update โ Service โ RDS โ user. Rollback is justgit revert. - Real orgs split these layers across owners; the boundaries double as blast-radius boundaries.
Self-check quiz¶
Pehle memory se jawab do, phir neeche kholo.
- Name the 8 bridges and one failure mode for each.
- Which bridge is the CIโCD seam, and why does CI not touch the cluster?
- Give the same golden thread (reconciliation) as it appears in four different tools.
- tfstate is deleted. What does the next
terraform planthink, and why is that dangerous? - Why is Argo's pull model considered more secure than CI-driven
kubectl apply(push)? - Someone runs
kubectl editon a live Deployment. Walk through what Argo does and which thread explains it. - Where does durable state live in this system, and what does that let you do with compute?
Jawab dekho
- B1 TFโAnsible: IPsโinventory; UNREACHABLE. B2 AnsibleโK8s: 3 playbooks live cluster banate hain; kubeadm fail / wrong join token. B3 DockerโPod: ECR image kubelet pull karta hai; ImagePullBackOff. B4 pushโCI: git push workflow trigger karta hai; wrong branch filter. B5โญ CIโGit manifest: image tag update+commit, cluster seedha nahi; manifest update bhool gaye โ old image chalta rehta. B6 GitโArgoโK8s: Argo pull+apply karta hai; OutOfSync sync nahi hua. B7 PodโServiceโUser: EndpointSlice ready pods route karta hai; koi ready pods nahi โ 503. B8 PodโRDS: psycopg2 se RDS:5432 connect; wrong SG ya bad password โ timeout.
- Bridge 5. CI cluster ko seedha nahi chhuta โ sirf Git mein manifest update karta hai. Argo (cluster ke andar) Git se pull karke apply karta hai. Isliye CI ke paas cluster credentials nahi chahiye.
- Terraform:
apply.tf files vs tfstate/live AWS compare karta hai. Ansible: playbookstate=presentvs host reality. Kubernetes: Deployment spec vs live ReplicaSet/Pod count, continuously. Argo CD: Git manifest vs live cluster, har ~3 min. - Terraform koi state nahi dekhta โ sochta hai kuch bhi provisioned nahi โ
plan"sab kuch create karo" kehta hai โ apply karo toh duplicate VPC/EC2/RDS banane ki koshish โ name conflicts ya costly duplicates. State file = Terraform ki yaadaasht. - Pull (Argo): creds cluster ke andar hain, Argo Git ko outbound read karta hai โ CI ke paas kuch nahi. Push (CI kubectl): CI ke paas kubeconfig โ CI hack = cluster gaya.
- Argo OutOfSync detect karta hai (Cause B โ cluster drifted). selfHeal:true hai toh Git manifest wapas apply karta hai, manual edit revert. Thread 1 (reconciliation) โ current state ko desired state tak drive karo, hamesha.
- Durable state: tfstate โ S3; DB โ RDS (cluster ke bahar). Yeh compute ko disposable banata hai โ pods aur servers Cattle hain, kill and replace freely, koi data loss nahi.
Hands-on lab¶
On paper (no cloud spend), from memory:
1. Draw the two loops and label which tools live in each.
2. Draw all 8 bridges as arrows; annotate each arrow with its artifact and failure mode.
3. Pick one golden thread and write the one line proving it appears in Terraform, Ansible, Kubernetes, and Argo.
4. Trace a single git push all the way to a user's browser, naming every bridge it crosses.
5. Now trace a rollback and a selfHeal event. Compare โ which threads does each use?
โ Sahi hua to aisa dikhega: If you can do all five from memory, you're ready for a systems-design interview.
Interview questions¶
- "Walk me through what happens from
git pushto the change being live for users." (This chapter, section 4 โ the whole answer.) - "CI or Argo โ who applies to the cluster? Why is that separation valuable?"
- "What's the difference between drift and lost state, and how does each get fixed?" (Thread 1 + Thread 2; see M1.)
- "Your Terraform state file is gone. What's your recovery plan and what's the blast radius?"
- "Why do we keep the database in RDS instead of a Kubernetes pod?" (Thread 2 โ state outside; see M4.)
- "Explain push vs pull deployment models and their security tradeoff." (Thread 4.)
Production challenge¶
A deploy went out 20 minutes ago. Users report 502s. You have: green CI, Argo showing Synced/Healthy, pods Running. Using the bridges as a checklist, write the ordered sequence of checks you'd run โ from Bridge 7 (ServiceโPod, is anything in the EndpointSlice?) back through Bridge 6 (did Argo actually apply the intended image?) to Bridge 8 (can the pod reach RDS?). Name the single command you'd run at each bridge. (Cross-reference the per-hop debug commands in M9 ยงThe full instrumented request lifecycle.)
Next โ M8 โ Observability & SRE: now that you can draw the system, learn to see inside it while it's running.