Configuration drift

Configuration drift is the difference that builds up between how a system is declared, in code or a runbook, and how it is really configured. It comes from hotfixes in a console, manual package upgrades and one-off scripts. Over time servers that should be identical stop matching, and the code no longer describes production.

How it works

Drift is found by comparing declared state with live state. With infrastructure as code, terraform plan -refresh-only shows what changed outside Terraform. Say an engineer opened port 22 on a security group during an incident. The next normal terraform plan proposes removing that rule, and an apply could break whatever came to depend on it. In GitOps, an agent compares the cluster with Git every few minutes and flags or reverts manual edits.

Why it matters in production

Drift makes changes unpredictable, because the plan you reviewed is not the system you are changing. It also breaks disaster recovery, since rebuilding from code gives you a different system from the one that was running. Detect it on a schedule and route every fix back through code.

See also

• Infrastructure as code

• GitOps

• Container image

• Disaster recovery

On the One2N blog

• GitOps for Kafka in the real world: how we governed 78 clusters without breaking production

• Terraform: best practices and gotchas

• How temporal versioning prevents configuration drift in stream processing

Further reading

• Terraform tutorial: manage resource drift