kube-prometheus-stack 安装

kube-prometheus-stack 安装

kube-prometheus-stack 是 prometheus 的官方 helm charts,包含 prometheus-operatorprometheusgrafanaalertmanagernode-exporter 等组件。

安装 kube-prometheus-stack #

使用 helm charts 安装 kube-prometheus-stack

mkdir -p ~/charts/kube-prometheus-stack
cd ~/charts/kube-prometheus-stack
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
# values.yaml 用来查看默认值
helm show values prometheus-community/kube-prometheus-stack > values.yaml
cat <<EOF > custom-values.yaml
prometheus:
  prometheusSpec:
    additionalScrapeConfigs: []
    podMonitorSelectorNilUsesHelmValues: false
    ruleSelectorNilUsesHelmValues: false
    probeSelectorNilUsesHelmValues: false
    scrapeConfigSelectorNilUsesHelmValues: false
    serviceMonitorSelectorNilUsesHelmValues: false
    storageSpec:
      volumeClaimTemplate:
        spec:
          storageClassName: "ceph-block"
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 50Gi
  service:
    type: NodePort
  ingress:
    enabled: true
    ingressClassName: nginx
    hosts: ['prometheus.lan']
  replicas: 1
  retention: 10d

prometheusOperator:
  enabled: true

# grafana service
grafana:
  service:
    type: NodePort
  ingress:
    enabled: true
    ingressClassName: nginx
    hosts: ['grafana.lan']
  persistence:
    enabled: true
    type: sts
    storageClassName: "ceph-block"
    accessModes:
      - ReadWriteOnce
    size: 20Gi

alertmanager:
  enabled: true

nodeExporter:
  enabled: true
EOF
helm upgrade --install --create-namespace --namespace monitoring kube-prometheus-stack prometheus-community/kube-prometheus-stack -f custom-values.yaml

配置说明 #

有一个相关 issue 讨论:servicemonitor not being discovered

prometheus:
  prometheusSpec:
    podMonitorSelectorNilUsesHelmValues: false
    ruleSelectorNilUsesHelmValues: false
    probeSelectorNilUsesHelmValues: false
    scrapeConfigSelectorNilUsesHelmValues: false
    serviceMonitorSelectorNilUsesHelmValues: false

如果没有配置上面这些,在相关 selector 为空时,就会使用 release: kube-prometheus-stack 作为默认的,自己创建的 ServiceMonitor 等资源如果没有设置此 label 就会被会被自动服务发现。比较坑人!!!

# kubectl -n monitoring get prometheus -oyaml
apiVersion: v1
items:
- apiVersion: monitoring.coreos.com/v1
  kind: Prometheus
    name: kube-prometheus-stack-prometheus
    namespace: monitoring
  spec:
    podMonitorNamespaceSelector: {}
    podMonitorSelector:
      matchLabels:
        release: kube-prometheus-stack
    probeNamespaceSelector: {}
    probeSelector:
      matchLabels:
        release: kube-prometheus-stack
    ruleNamespaceSelector: {}
    ruleSelector:
      matchLabels:
        release: kube-prometheus-stack
    scrapeConfigNamespaceSelector: {}
    scrapeConfigSelector:
      matchLabels:
        release: kube-prometheus-stack
    serviceMonitorNamespaceSelector: {}
    serviceMonitorSelector:
      matchLabels:
        release: kube-prometheus-stack

配置 grafana dashboard #

默认账号密码:

账号:admin
密码:prom-operator

导入一个查看 node exporter 的 dashboard

https://grafana.com/grafana/dashboards/16098-node-exporter-dashboard-20240520-job/