[prometheus-users] alertmanager send pagerduty alerts resulution with 3 min delay

2020-08-12 Thread Radosław Ciok
Hello, I have problem that alertmanager resolves pagerduty alerts with 3 min delay. Alerts themselves come to PD exactly on time where fired. Does someone know where this delay might be coming from ? as for config I don't use any grouping on alertmanager side. For resolve I set only

Re: [prometheus-users] Grouping of alarms (group_interval, group_wait and repeat_interval)

2020-08-12 Thread Christian Hoffmann
Hi, On 8/12/20 3:41 PM, rosaLux161 wrote: > If alert 1 and alert 2 occur simultaneously or in a very short time, > then only one alert should be sent out. If alert 2 only occurs after > some time, then another alert should be sent. The latter does not work. > If alert 2 occurs, nothing happens.

Re: [prometheus-users] Filter on range query

2020-08-12 Thread Johny
In this case, I am doing a remote read. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this

Re: [prometheus-users] Filter on range query

2020-08-12 Thread Julien Pivotto
On 12 Aug 09:12, Johny wrote: > I have an alert condition based on count of metric value > 0 in last 10 min > below a certain threshold. > count_over_time((mymetric{..} > 0)[10m:1m]) > x > > Problem with this query is it evaluates each min in last 10 min and returns > total of 5 even if there

[prometheus-users] Filter on range query

2020-08-12 Thread Johny
I have an alert condition based on count of metric value > 0 in last 10 min below a certain threshold. count_over_time((mymetric{..} > 0)[10m:1m]) > x Problem with this query is it evaluates each min in last 10 min and returns total of 5 even if there was a single data point in db — because of

[prometheus-users] doubt with location of the component "servicemonitor"

2020-08-12 Thread Damian Lezcano
hello everyone! Does the "service monitor" component have to be in the same project as the services I want to monitor? That is, I have the "monitoring" project and the "fuse" services are in another project called "PROD", so the "servicemonitor" component must be in the "PROD" project. is that

[prometheus-users] Grouping of alarms (group_interval, group_wait and repeat_interval)

2020-08-12 Thread rosaLux161
I'm trying to understand how the grouping of alarms works. The alertmanager is configured as follows: ``` --- route: group_by: - alertname group_interval: 10s group_wait: 10s receiver: opsgenie repeat_interval: 1m routes: - receiver: opsgenie receivers: - name: opsgenie opsgenie_configs: -

Re: [prometheus-users] Re: Alertmanager to slack not working

2020-08-12 Thread Brian Candler
It logs to stdout/stderr. If you're running alertmanager under systemd then try "journalctl -eu alertmanager". Or simply stop it, and then run it in the foreground in a terminal. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To

[prometheus-users] High CPU consumption by JMX exporter

2020-08-12 Thread Brajesh Shishir
I am using JMX exporter for kafka related metrics. We are running it as javagent as shown below:- export KAFKA_OPTS='-javaagent:/jmx/jmx_prometheus_javaagent-0.12.0.jar=7097:/kafka/kafka-jmx-metrics.yaml' It shows high cpu usage(90%-100%) for a period of 20 minutes approx. once every day. Is

Re: [prometheus-users] Re: Alertmanager to slack not working

2020-08-12 Thread kiran
Thank you, Brian. I started from scratch and used the alertmanager.yml from this link: https://awesome-prometheus-alerts.grep.to/alertmanager I see its firing but not receiving on slack. Is there any log file for alert manager to debug? I am attaching as text file my alertmanager.yml On Wed, Aug

Re: [prometheus-users] PromQL stateless issue

2020-08-12 Thread fiala...@gmail.com
Thank you! It helps. On Wednesday, August 12, 2020 at 9:05:17 AM UTC+2 Julien Pivotto wrote: > On 12 Aug 00:00, fiala...@gmail.com wrote: > > Hi, > > > > I would like to write a rule for alerting e.g. packet discards in > percent. > > > > My requirements are: > > - trigger an alert when

Re: [prometheus-users] PromQL stateless issue

2020-08-12 Thread Julien Pivotto
On 12 Aug 00:00, fiala...@gmail.com wrote: > Hi, > > I would like to write a rule for alerting e.g. packet discards in percent. > > My requirements are: > - trigger an alert when discards are at least 10 minutes higher than X > - close alert when discards are more then 60m equals to zero > >

[prometheus-users] PromQL stateless issue

2020-08-12 Thread fiala...@gmail.com
Hi, I would like to write a rule for alerting e.g. packet discards in percent. My requirements are: - trigger an alert when discards are at least 10 minutes higher than X - close alert when discards are more then 60m equals to zero I'm not sure, how to define it in PromQL. Because I haven't

[prometheus-users] Re: Alertmanager to slack not working

2020-08-12 Thread Brian Candler
Your alertmanager config is broken. "receivers" needs to be a list, as does slack_configs, and the settings need to be indented. I doubt you're even able to start alertmanager with that config. You posted images rather than text, so I can't copy-paste your config to fix it, but here's a