Hi all (and sorry about my English level) I have а issue with Prometheus in k8s (deployed by Prometheus operator). After the start of the container, it is very slow and not responsive. Process prometeus consume 100-150% cpu (1-1.5 core vs 0.7 core on another, but identical cluster), and response take 5-20 seconds. LA is growing on the node to 7-10 but it is not a lack of resources on the node, other containers work fine. Prometheus container has no cpu and memory limits. Metrics targets are responsive.
Inside the container, the commands ps and top are also slowed down. Other commands work normal. [master1 ~]$ time curl http://10.233.67.83:9090 <a href="/graph">Found</a>. real 0m8.400s user 0m0.004s sys 0m0.014s I have tried different versions of the Prometheus, different versions of the Prometheus operator, it did not help. I have tried a deployment without the operator and got the same result. There is nothing useful in logs, even in debug mode. Can anyone help with diagnostics and debag methods? helm values.yaml: # helm install prom-operator stable/prometheus-operator --namespace monitoring -f values.yaml --version 9.3.1 prometheus: ingress: enabled: true ... prometheusSpec: # logLevel: debug retention: 30m image: tag: v2.20.1 externalLabels: k8s_cluster: "cluster1" -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/72a22ef7-81c1-435e-bdc0-60bd64b51dcan%40googlegroups.com.
