[prometheus-users] Kube-state-metrics not working
Hello @all Kubernetes Version: 1.23.7-gke.1400 Kube-state-metrics Version: 2.5.0 I am seeing an issue with kube-state-metircs on GKE. Suddenly it has stopped working. I compared manifets of clusterrole, clusterolebinding and serviceaccount and all looks good. Still e are getting below error. E0708 11:16:16.031316 1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.24.1/tools/cache/reflector.go:167: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:monitoring:prometheus-kube-state-metrics" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope Similar error for all other k8s resources. Does any know any related issues/solutions? Thanks -- The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify ATMECS and delete it from your computer. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/70474ab4-e1a7-464b-9554-b44e647ffa73n%40googlegroups.com.
[prometheus-users] Prometheus keep restarting following WAL replay completion
Hi All, The WAL replay is continuing indefinetly while increaseing the maxsegment. We are using Prometheus 2.25.0 Logs are attached as the image. [image: Screenshot from 2021-10-21 09-45-29.png] Is this a known Bug? or Do we ahve any steps to fix this? Thanks Nishant -- The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify ATMECS and delete it from your computer. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/daac9a87-bf63-40c9-ad05-c7f777193c9cn%40googlegroups.com.
[prometheus-users] Prometheus Adapter Custom Metrics
Hi All,
I am creating a custom metrics for HPA. I want to scale on the based of
nginx_ingress_controller_requests. But I am getting 404 in adapter logs as
below.
I0121 15:21:10.089817 1 httplog.go:89] "HTTP" verb="GET"
URI="/apis/custom.metrics.k8s.io/v1beta1/namespaces/dev/ingresses.extensions/dev-dotcom-ingress/nic_requests"
latency="3.573968ms" userAgent="vpa-recommender/v0.0.0 (linux/amd64)
kubernetes/$Format/kuba-horizontal-pod-autoscaler" resp=404
Following is my custom rule.
custom:
- seriesQuery: 'nginx_ingress_controller_requests'
resources:
overrides:
ingress: {resource: "ingress"}
name:
as: "nic_requests"
metricsQuery:
'sum(rate(nginx_ingress_controller_requests{ingress!="",status!="503",status!="504"}[1m]))
by (<<.GroupBy>>)'
Appreciate any help, Thanks a lot.
--
The information in this message may be proprietary and/or confidential, and
protected from disclosure. If the reader of this message is not the
intended recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited. If
you have received this communication in error, please notify ATMECS and
delete it from your computer.
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/8fecd0f1-463c-4361-8c59-068d4799f6bcn%40googlegroups.com.
[prometheus-users] Re: Node exporter was down
Hi, It's better if you run the node_exporter as a system service. Then you can check the node_exporter logs too. https://kifarunix.com/monitor-linux-system-metrics-with-prometheus-node-exporter/ Thanks On Monday, November 2, 2020 at 7:55:25 PM UTC+5:30 srika...@avankia.com wrote: > Hi guys, > i am running Node Exporter with (./nodeexporter &) command > but it was showing down after sometime, but the server memory and CPU was > fine > Can someone help me out there, how can i overcome this problem > and also i would like to know how can i check these node-exporter logs > > > Thanks In Advance > > -- The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify ATMECS and delete it from your computer. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6782a069-9492-453d-9e11-ba1e646fdaafn%40googlegroups.com.
[prometheus-users] Monitoring Master-Slave connection with Prometheus
Hi all, I am trying to figure out how can I monitor the connection between the Salt master and minion. We are trying to set up alerts if the connection breaks for any reason. Any help will be appreciated. Thanks -- The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify ATMECS and delete it from your computer. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/9e22a623-5d4e-488b-82c3-b5cc6e327a6cn%40googlegroups.com.
