[prometheus-users] Unable to expose through istio ingress gateway

I am unable to expose the pushgateway through istio ingress gateway. I have tried to expose through a the path /pushgateway and through a sub domain.. helm install is helm install prometheus-pushgateway \ --set ingress.enabled=true \ --set ingress.hosts[0]="loadbalancer.dns.com" \ --set

[prometheus-users] suggest document for alert manager

Hi, I am working on alert-manager , very new to this . I want to write query for if data is not received on certain topic/feed within one hour , it should send mail to a particular group , such kind of alerting mechanism I have to implement . Please suggest any documentation I can follow .

[prometheus-users] Re: black box exporter basic auth

This has been discussed before. Anything you can use to obfuscate the password, in a way that prometheus itself could unobfuscate it at startup, would also be usable by an attacker who has root access to the system. The best I can offer is to have the entire config file gpg-encrypted, decrypt

Re: [prometheus-users] Re: How to optimize prometheus memory for best perfromance and prevent pods from crashes due to memory.

On Friday, 24 April 2020 20:37:14 UTC+1, Ravi Teja Reddy Bonthu wrote: > > Can you pls help me with the thumb rule for Storage and Memory based on > what calculation i can find what could be my memory and storage and the > resource limits (CPU memory ) that i can set for the prometheus pods in K

[prometheus-users] Re: Recording the hostnames of all targets.

On Friday, 24 April 2020 10:09:44 UTC+1, Yagyansh S. Kumar wrote: > > UP metrics will exist but if the node exporter itself is down, it > won't expose the metric at that time right? So, I won't get the "nodename" > label from node_uname_info. > That's correct. You could at least alert on th

[prometheus-users] Re: Is there any way to do count_values_over_time?

Thanks Brian! I believe that works for me. You are amazing! -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. T

[prometheus-users] black box exporter basic auth

hi, I am using basic auth in Prometheus black box exporter yml files. basic_auth: username: username password: password123 Is there a way to avoid a clear text password, some way to hash it out, etc. please help. I am in a prod environment so I need to strictly avoid

Re: [prometheus-users] Re: How to optimize prometheus memory for best perfromance and prevent pods from crashes due to memory.

Thank you Brian Candler ! I have upgraded all my prometheus-operator-0.38.0, prometheus-2.17.1 , alertmanager, grafana, node-exporter and kube-state-metrics to their latest versions, i can see those stats and what you said is absolutely correct i.e under Status > Runtime & Build Information. I

Re: [prometheus-users] Node Exporter have wrong nodename in node_uname_info.

Got it now, thanks once again, Julius. On Friday, 24 April 2020 18:47:55 UTC+5, Julius Volz wrote: > > On Fri, Apr 24, 2020 at 2:37 PM Sohaib Omar > wrote: > >> Hi all, >> I am stuck in some weird situation. in metric *node_uname_info *I get >> the same value for nodename and pod label. >> > >

[prometheus-users] MongoDB Enterprise with Prometheus

Is it necessary an exporter to monitor MongoDB Enterprise? I ask this because I know MongoDB Enterprise has an Ops Manager. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, se

Re: [prometheus-users] Most of the time metrics are coming as empty

Thanks Julius for the info and confirming about the approach. On Friday, April 24, 2020 at 12:38:25 PM UTC+5:30, Julius Volz wrote: > > To be more precise about maximum scrape times: an instant vector selector > in PromQL (e.g. foo{bar="baz"}) looks back a maximum of 5 minutes from the > curren

Re: [prometheus-users] Most of the time metrics are coming as empty

*Thanks Brian Candler, for your valuable suggestion. We will try this also.* On Friday, April 24, 2020 at 2:14:24 PM UTC+5:30, Brian Candler wrote: > > On Friday, 24 April 2020 06:57:08 UTC+1, Srinivasa praveen wrote: >> >> Thanks for the response Stuark. The reason behind keeping the scraping

[prometheus-users] Re: Prometheus alerts with a ($) value that is below the threshold

Is there a way to print the values of the parts as well, so e.g. the value node_memory_MemTotal_bytes? Or is it possible to have the evaluation steps logged? Op vrijdag 24 april 2020 11:18:28 UTC+2 schreef J Houer: > > We use a Prometheus alert (and node-exporter) to check whether we are > ru

Re: [prometheus-users] Prometheus alerts with a ($) value that is below the threshold

This is very strange, to say the least. With the >= 90 filter on the top level of the expression, the alerting rule should never return series that have a sample value below 90, and thus $value should never be below 90 either. In case this is really not a misconfiguration somewhere (please double-

Re: [prometheus-users] Prometheus snapshot restore doesn't show the old metrics

Did you set the retention (--storage.tsdb.retention.time) to long enough on your new Prometheus instance, so that the old data doesn't slide out of the retention window? On Fri, Apr 24, 2020 at 2:29 PM KARIM MANAOUIL wrote: > Hi all, > > I am doing analysis of a Kubernetes cluster and because I

Re: [prometheus-users] Node Exporter have wrong nodename in node_uname_info.

On Fri, Apr 24, 2020 at 2:37 PM Sohaib Omar wrote: > Hi all, > I am stuck in some weird situation. in metric *node_uname_info *I get the > same value for nodename and pod label. > That is to be expected, because uname's "nodename" refers to the hostname, and Kubernetes by default sets a pod's ho

[prometheus-users] no main manifest attribute, in jmx_prometheus_javaagent-0.12.0.jar

Hello guys, I want to setup JMX exporter in my linux system and I am following https://github.com/prometheus/jmx_exporter. I have download the jmx_prometheus_javaagent-0.12.0.jar also version 0.11.0 and also created config.yml file in my linux system. Java 1.8 is installed in my linux system.

[prometheus-users] Node Exporter have wrong nodename in node_uname_info.

Hi all, I am stuck in some weird situation. in metric *node_uname_info *I get the same value for nodename and pod label. node_uname_info{domainname="(none)",endpoint="metrics",instance= "x.x.x.x:9100",job="node-exporter",machine="x86_64",namespace="monitoring", nodename="node-exporter-ljhlp",po

Re: [prometheus-users] Prometheus snapshot restore doesn't show the old metrics

Hello, from the article you linked, older blocks are only snapshotted as hardlinks while open blocks are dumped. Maybe you didn't copy the hardlinks correctly? check the size of your folder On Fri, Apr 24, 2020 at 2:29 PM KARIM MANAOUIL wrote: > Hi all, > > I am doing analysis of a Kubernetes cl

[prometheus-users] Prometheus sending logs to BEATS input in Graylog

Hi, I have deployed a Graylog instance inside k8s along with Prometheus. The logs from the k8s cluster are collected by fluentbit and send to the graylog server, but however I see Prometheus trying to send constantly logs to Graylog straight and those failing: 2020-04-24 12:33:05,562 ERROR

[prometheus-users] Prometheus snapshot restore doesn't show the old metrics

Hi all, I am doing analysis of a Kubernetes cluster and because I don't have permanent access to it, I thought about this idea of importing Prometheus TSDB and analyze the data using a local Prometheus instance. I succeeded to do that using snapshots as told here

Re: [prometheus-users] How to scrape opendj,tomcat and openam processes using process exporter in prometheus.

As far as I know you cannot *select* by the process ID, but you can add the PID to the final group name label via: process_names: - name: "{{.Comm}}-{{.PID}}" cmdline: - 'openam|opendj|tomcat' See https://github.com/ncabatoff/process-exporter#using-a-config-file-group-name That way y

Re: [prometheus-users] Re: Blackbox exporter probe_ssl_earliest_cert_expiry giving wrong value in grafana

Hi Per thanks for the input, issue was already resolved, i was able to configured and make some changes on SSL Monitoring, thank you guys Regards, James On Friday, April 24, 2020 at 7:25:26 PM UTC+8, Per Carlson wrote: > > Ok noted, so is there any work around to add the request like display the

Re: [prometheus-users] Re: Blackbox exporter probe_ssl_earliest_cert_expiry giving wrong value in grafana

> > Ok noted, so is there any work around to add the request like display the > actual expiration based on the certificate actual status? > You could try https://github.com/ribbybibby/ssl_exporter to track the expiry of every certificate in the chain. But as many have told you in this thread, you

[prometheus-users] Prometheus alerts with a ($) value that is below the threshold

We use a Prometheus alert (and node-exporter) to check whether we are running out of memory on a node. Issue: In many cases I get an alert with a $value that is below the threshold value in the expression.The expression is: alert: GettingOutOfMemory > expr: max(sum > by(instance) n

[prometheus-users] Re: Recording the hostnames of all targets.

Thanks Brain. > > Can you give some more specific examples? What metric are you joining > with - perhaps node_uname_info? >> > - alert: HighCpuLoadCrit expr: (node_load15 > (2 * count without (cpu, mode) (node_cpu_seconds_total{mode="system"}))) ** on(instance) group_left(nodenam

[prometheus-users] Re: Manage prometheus configuration using Prometheus operator

About alertmanager configuration,you can find alertmanager`s secret ,and using base64 decoder data you could look alertmanager`s configing.So you can delete it and create new one by your custom configuration. And another configuration,for example alert rule,you could use this command '

Re: [prometheus-users] Most of the time metrics are coming as empty

On Friday, 24 April 2020 06:57:08 UTC+1, Srinivasa praveen wrote: > > Thanks for the response Stuark. The reason behind keeping the scraping > interval so long is, on receiving scrape request from Prometheus, my > exporter performs around 10 queries against database and exposes the result > as 1

[prometheus-users] Re: Recording the hostnames of all targets.

> I cannot have the hostname as Node Exporter won't be scraping any metrics at that point. Can you give some more specific examples? What metric are you joining with - perhaps node_uname_info? Note that the "up" metric will still exist (with a value of 0) when a scrape fails - this means: (a)

Re: [prometheus-users] How to scrape opendj,tomcat and openam processes using process exporter in prometheus.

Thanks for your reply Julius. One quick question can i pass the process id too , if yes how ? On Fri, Apr 24, 2020 at 1:25 PM Julius Volz wrote: > You should be able to just edit the "cmdline" regex part to match only the > processes that you mentioned: > > process_names: > - name: "{{.Comm}}

Re: [prometheus-users] How to scrape opendj,tomcat and openam processes using process exporter in prometheus.

You should be able to just edit the "cmdline" regex part to match only the processes that you mentioned: process_names: - name: "{{.Comm}}" cmdline: - 'openam|opendj|tomcat' Depending on the exact names of the command-lines you may need to adjust the regex or use one of the other sele

[prometheus-users] Re: How can i add different email template in my alertmanager for different alerts.As i need different subject line for each alert.

When i am using subject it adds the subject line on the mail,that works but the html part which i have added is not working .I dont get mail with previous html format with updated subject line. - name: Monitoring email_configs: - to: 'pooja.ch...@gmail.com' html: '{{ templ

Re: [prometheus-users] Most of the time metrics are coming as empty

To be more precise about maximum scrape times: an instant vector selector in PromQL (e.g. foo{bar="baz"}) looks back a maximum of 5 minutes from the current evaluation timestamp to find the latest sample for each matched series before that timestamp. If the last sample is more than 5 minutes old, n