I am trying to target and scrape node-exporter and Kube-state-metric pods
running in the EKS cluster using Prometheus. For service discovery, I am
using kubernetes_sd_configs but Prometheus is not able to target the
required pods.
Node-exporter and Prometheus are running in the same namespace while
Kube-state-metrics is running in another namespace.
I am using pod annotations to discover the pods.
Following is the snippet of node-exporter.yml, prometheus.yml
Node-exporter.yaml:
apiVersion: apps/v1
kind: DaemonSet
metadata:
labels:
app.kubernetes.io/component: exporter
app.kubernetes.io/name: node-exporter
name: node-exporter
namespace: amazon-cloudwatch
spec:
selector:
matchLabels:
app.kubernetes.io/component: exporter
app.kubernetes.io/name: node-exporter
template:
metadata:
annotations:
prometheus.io/scrape: true
prometheus.io/path: /metrics
prometheus.io/port: "9100"
prometheus.io/scheme: "http"
labels:
app.kubernetes.io/component: exporter
app.kubernetes.io/name: node-exporter
The same goes for the Kube-state-metrics.yml file only with minor changes
but not in the annotations part. As for Prometheus, my code snippet of
Prometheus scrape configs is follow:
- job_name: "kubernetes-pods"
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels:
[__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels:
[__meta_kubernetes_pod_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__address__,
__meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
regex: (.+):(?:\d+);(\d+)
replacement: ${1}:${2}
target_label: __address__
- source_labels:
[__meta_kubernetes_pod_annotation_prometheus_io_scheme]
action: replace
target_label: __scheme__
regex: (.+)
Can anyone please point out any error I am doing because the above snippets
are not working. Also suggestions for using some other configurations are
also appreciated.
Thanks
Ansh Lehri
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/1ecaec40-ad65-46d0-8995-e80288aba5f1n%40googlegroups.com.
