As far as I can see, you haven't shown your actual alerting rule.
However, it's straightforward to debug this: paste your entire alerting
"expr" into the PromQL query interface. Wherever the line is present, it
means an alert will fire. You can then work backwards from that to find
the problem with your expr.
For example, say you have this rule:
expr: avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[2m]))
< 0.8
Paste exactly "avg by (instance)
(rate(node_cpu_seconds_total{mode="idle"}[2m])) < 0.8" into the PromQL
browser to see if and when it fires.
In PromQL, the expression "foo" generates a vector: the set of all
timeseries whose metric name is "foo". Then "foo < 0.8" is a filter, not a
boolean. It filters the vector to only those whose value is less than
0.8. When used as an alerting expression, you get an alert if the vector
is not empty.
On Sunday, 20 February 2022 at 16:38:10 UTC [email protected]
wrote:
> Hello everybody.
> We are facing some issues with CPU monitoring.
> Our graphs don't show reaching the thresholds even one time, not for 3m.
> All info and screenshots will be below.
> Alert is configured to alert at 20%. Related only to the blue graph.
>
> [image: Screenshot 2022-02-18 133538.png]
>
> [image: Screenshot 2022-02-18 133642.png]
>
> Prometheus creates a massive amount of alerts in our Opsgenie, there are
> no issues with other alerts or even with a threshold of 60%.
> [image: Screenshot 2022-02-18 133820.png]
>
> Alert query:
>
> [image: Screenshot 2022-02-18 134142.png]
>
> Maybe you have some suggestions on what can cause that flapping and
> triggering the alert?
> Already tried to check graphs by 1,2,5,10 minute, by the hour and etc,
> there is nothing that should result in an alert.
> Also, there are no such alerts from Cloudwatch monitoring.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/a34afe7e-b651-4553-afb8-ebf8436c91c0n%40googlegroups.com.
