Lazy loaded image
运维管理
helm drift 排查
Words 427Read Time≈ 2 min
2026-1-30
2026-2-1
date
related_level
slug
k8s_helm_drift
type
Post
relate_date
summary
helm 漂移现象以及排查方法
status
Published
tags
k8s
版本管理
category
运维管理
last_updated
Feb 1, 2026 05:20 PM
是否已更新
orginal_page
是否推荐
参考资料
  • Helm Releases
    Helm Releases

    Helm Releases

    The GitOps Toolkit Custom Resource Definitions documentation.

  • CNCFCNCFSolving configuration drift using GitOps with Argo CD
    Solving configuration drift using GitOps with Argo CD

    Solving configuration drift using GitOps with Argo CD

    Argo CD (part of the Argo project) is a deployment solution for Kubernetes that follows the GitOps paradigm. In the most basic scenario, Argo CD continuously monitors a Git repository with Kubernetes…

    CNCFCNCF
  • KomodorKomodorDrift Detection in Kubernetes
    Drift Detection in Kubernetes

    Drift Detection in Kubernetes

    Although IaC (and CaC) bring immense value, they can also lead to a major problem: configuration drift. In this article, we will take a closer look at this issue and explore different methods of keeping systems in their intended state.

    KomodorKomodor
helm 漂移是指集群状态与 Helm 存储中的清单相比存在的差异
  • 差异来自于手动干预
    • 对 Kubernetes 对象中的热修复，如 patch
    • 因故障、功能或安全问题而进行的手动更改
    • 控制器（HPA / Operator / Webhook）自动对 YAML 进行的更改
    • chart 升级、values 漏同步
  • 这些更改可能会被后续的 Helm 版本覆盖，从而导致配置丢失
排查 helm 漂移问题
（不推荐）手动获取 helm 渲染状态以及集群当前状态
  • helm template <release> <chart> \ -n <ns> \ -f values.yaml \ --debug > expected.yaml
  • helm get manifest <release> -n <ns> > expected.yaml
    • 如果没有 chart 源码
  • kubectl get all -n <ns> -o yaml > actual.yaml
  • diff 工具查看差异
    • diff -u expected.yaml actual.yaml
    • dyff between expected.yaml actual.yaml
      • 更美观
  • helm diff 查看是否存在漂移以及漂移的对应差异内容
    • helm diff upgrade <release> <chart> \ -n <ns> \ -f values.yaml \ --reuse-values
  • kubectl 查看 managedFields 字段确认修改的来源是 kubectl 还是控制器
    • kubectl get deploy <name> -n <ns> -o yaml | yq '.metadata.managedFields'
      • manager: kubectl-edit
      • manager: kube-controller-manager
      • manager: helm
  • kubectl 查看 events 确认具体修改事件
    • kubectl events -n <namespace> --for=pvc/test-pvc --types=Warning,Normal
      • 查看资源 pvc/test-pvc 的 warn, normal 级别的事件
    • kubectl get events -n <namespace> --field-selector involvedObject.name=test-pvc,involvedObject.kind=PersistentVolumeClaim,type=Warning
      • 查看名称为 test-pvc，kind 为 pvc 的资源 warn 级别事件
避免 helm 漂移
  • 遵循 GitOps 准则，把 Git 作为管理基础设施和应用程序配置的单一数据源，避免手动更改
    • RBAC 禁止 kubectl edit/apply
    • 只允许 Helm / GitOps SA
  • 通过 Argo CD / Flux 自动对齐期望状态
  • Drift 定期巡检并报警
    • helm diff upgrade ... || alert
上一篇
WSL 安装和配置
下一篇
yaml 文件比较工具 jd, dyff
Comments
Loading...