Hi Stuart. Thanks for the quick response. But I am using the current value
in Annotations only.
Here is an example of my Alert Rule:
- alert: HighCpuUtilisationCrit
expr: (sum by (instance,node,cluster)
(sum(irate(node_cpu_seconds_total{mode!~"idle"} [5m])) without (cpu) /
count(node_cpu_seconds_total{mode!~"idle"}) without (cpu) * 100) > 85) * on
(instance) group_left(nodename) node_uname_info
for: 2m
labels:
severity: "CRITICAL"
annotations:
summary: "CPU Utilisation on *{{ $labels.instance }}* - *{{
$labels.nodename }}* is more than 90%."
description: "Current Value = *{{ $value | humanize }}*"
identifier: "*Cluster:* `{{ $labels.cluster }}`, *node:* `{{
$labels.node }}` "
On Tuesday, April 14, 2020 at 4:41:43 PM UTC+5:30, Stuart Clark wrote:
>
> On 2020-04-14 12:04, Yagyansh S. Kumar wrote:
> > Hi. I am using Alertmanager version 0.16.0. The Resolved Alerts that I
> > am receiving are wrong. The Alertmanager fires the resolved alert as
> > soon as the value decreases even slightly i.e it does not wait for the
> > value to get less than the threshold. And this thing is happening for
> > every alert.
> >
> > Example:
> > [1] Resolved
> >
> > Labels
> >
> > alertname = HighCpuUtilisationCrit
> >
> > cluster = ANALYTICS
> >
> > instance = 172.20.8.186:9100
> >
> > description = Current Value = *95.73*
> >
> > summary = CPU Utilisation on *172.20.8.186:9100* -
> > *dh4-k2-analytics-ga-ping-n1.dailyhunt.in* is more than 90%.
> >
> > Here, my threshold is 90% but I am receiving the resolved alert at
> > 95.73%.
> > Can someone help?
>
> That looks like you are putting the current value in a label. As a
> result any time it changes a new alert will be created. Try moving that
> to an annotation instead.
>
> --
> Stuart Clark
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/7fd47790-4892-4707-af32-aa3c7069a3f2%40googlegroups.com.
