The Rise of GitOps: ArgoCD and Flux
GitOps is changing how we think about Kubernetes deployments by making Git the single source of truth for cluster state
There is a pattern emerging in the Kubernetes ecosystem that I believe will become the default way teams deploy applications within two years. It is called GitOps, a term coined by Weaveworks in 2017, and after spending several months adopting it across our infrastructure, I am convinced it is the most significant operational improvement we have made since moving to Kubernetes itself.
The core idea is deceptively simple: Git is the single source of truth for your desired cluster state. Every change to your infrastructure or application deployment goes through a Git commit. An operator running inside the cluster watches the Git repository and continuously reconciles the actual cluster state with the desired state defined in Git.
No more kubectl apply from laptops. No more CI pipelines pushing directly to clusters. No more "who changed what and when" mysteries. Git has the answer to all of those questions because Git is the only way changes happen.
Why Traditional CI/CD Falls Short
The traditional approach to Kubernetes deployment looks like this: a CI pipeline builds your container image, pushes it to a registry, and then runs kubectl apply or helm upgrade to deploy it to the cluster. This works, but it has fundamental problems.
Push-based access. Your CI system needs credentials to talk to the Kubernetes API server. Those credentials are typically stored as secrets in your CI tool, and they need broad permissions to create and modify resources. Every CI pipeline that deploys to the cluster is a potential attack vector. If your Jenkins or CircleCI instance is compromised, your cluster is compromised.
Drift detection is nonexistent. Someone runs a manual kubectl edit to hotfix a production issue at 2 AM. The cluster state now diverges from what the CI pipeline last deployed. There is no automated mechanism to detect or correct this drift. The cluster state is whatever happened to it last, and you hope that was your pipeline.
Auditability gaps. CI pipeline logs tell you what was deployed, but correlating those logs with actual cluster state over time is painful. Git history, by contrast, is a complete, immutable audit trail.
Enter ArgoCD
We evaluated both ArgoCD and Flux, the two leading GitOps operators for Kubernetes. We chose ArgoCD for several reasons, though both tools implement the same fundamental pattern.
ArgoCD runs inside your cluster and watches one or more Git repositories. You define "Applications" that map a path in a Git repository to a namespace in the cluster. ArgoCD polls the repository (or receives webhook notifications), compares the desired state in Git against the actual state in the cluster, and reconciles any differences.
The architecture looks like this:
[Git Repository]
|
| (poll/webhook)
v
[ArgoCD UI] <== [ArgoCD Controller] ==> [Kubernetes API Server]
The controller runs a continuous reconciliation loop. If someone manually modifies a resource in the cluster, ArgoCD detects the drift and can either alert or automatically revert the change, depending on your configuration. We run with auto-sync enabled in non-production environments and manual sync with auto-revert in production.
Installation is straightforward:
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
Defining an application:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: payment-service
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/org/k8s-manifests.git
targetRevision: HEAD
path: apps/payment-service/production
destination:
server: https://kubernetes.default.svc
namespace: payment-service
syncPolicy:
automated:
prune: true
selfHeal: true
The selfHeal: true setting is the drift correction. If someone runs kubectl delete on a resource that should exist, ArgoCD recreates it. If someone kubectl edits a deployment, ArgoCD reverts the change. The Git repository is the authority, always.
Repository Structure
The repository structure matters more than most teams realize when adopting GitOps. We settled on a pattern that separates application manifests by environment and uses Kustomize for environment-specific overrides.
k8s-manifests/
apps/
payment-service/
base/
deployment.yaml
service.yaml
hpa.yaml
kustomization.yaml
staging/
kustomization.yaml # patches for staging
production/
kustomization.yaml # patches for production
catalog-service/
base/
staging/
production/
platform/
monitoring/
ingress/
cert-manager/
The base directory contains the canonical manifests. Environment directories contain only the differences: replica counts, resource limits, environment variables, ingress hostnames. Kustomize merges them at deploy time.
This structure gives us a clear separation between what an application is (base) and how it runs in each environment (overlays). It also means promoting a change from staging to production is a Git operation: update the image tag in the production kustomization and submit a pull request.
The Pull Request Workflow
This is where GitOps truly shines. Deploying to production becomes a pull request:
- CI builds a new container image, tags it with the git SHA, pushes it to the registry.
- CI opens a pull request against the manifests repository, updating the image tag.
- A team member reviews the change. The diff is small and clear: one line changed, the image tag.
- The pull request is merged.
- ArgoCD detects the new commit and syncs the cluster.
Every production deployment has a code review. Every deployment is traceable to a specific commit, a specific reviewer, a specific approval. The audit trail is built into the workflow, not bolted on as an afterthought.
Rollback is equally elegant: revert the Git commit. ArgoCD sees the revert, syncs the previous state, done. No need to remember which Helm chart version was running before, no digging through CI logs to find the last good build.
Flux: The Alternative
Flux, developed by Weaveworks, takes a slightly different approach. Where ArgoCD uses explicit Application custom resources, Flux uses annotations and custom resources that feel more Kubernetes-native. Flux also has tighter integration with Helm through its HelmRelease custom resource.
The choice between ArgoCD and Flux often comes down to preference. ArgoCD has a richer UI and better multi-cluster support out of the box. Flux is more lightweight and follows the Kubernetes operator pattern more closely. Both are CNCF projects (Flux is a sandbox project, ArgoCD is moving toward incubation) and both have active communities.
We chose ArgoCD primarily because the UI made it easier for teams who were new to GitOps to understand what was happening. Visibility into sync status, diff visualization, and deployment history in a web interface lowered the adoption barrier significantly.
Challenges We Encountered
Secret management. Storing secrets in Git is obviously unacceptable, but GitOps requires everything to be in Git. We adopted Sealed Secrets, which encrypts secrets with a cluster-specific key. The encrypted form can safely live in Git; only the controller inside the cluster can decrypt them.
Image tag updates. Automating the "update the image tag in the manifests repo" step required custom tooling. We wrote a small service that listens for container registry events and opens pull requests against the manifests repository. This is an area where the ecosystem is still maturing.
Learning curve. Engineers accustomed to kubectl apply needed time to internalize that they should never touch the cluster directly. Old habits die hard. We enforced this with RBAC: only ArgoCD's service account has write access to application namespaces. Human users have read-only access to production.
The Operational Impact
After three months of running GitOps with ArgoCD, the results are measurable. Deployment frequency increased because the barrier to deploying dropped from "run a pipeline and watch the logs" to "merge a pull request." Mean time to recovery decreased because rollback became a git revert. And the number of configuration drift incidents dropped to zero, because drift is automatically corrected.
GitOps is not just a deployment strategy. It is an operational philosophy that treats infrastructure the same way we treat application code: versioned, reviewed, tested, and automatically applied. For teams running Kubernetes at scale, I believe it will become table stakes within a year or two.