> When my alarm is firing, i would like auto-resolved it
Alerts are generated by a PromQL expression ("expr:"). For as long as this
returns a non-empty instance vector, the alert is firing. When the result
is empty, the alert stops.
For example: I want to get an alert whenever the metric
"megaraid_pd_media_errors" increases by more than 200. But if it has been
stable for 72 hours, I want the alert to go away. This is what I do:
- alert: megaraid_pd_media_errors_rate
expr: increase(megaraid_pd_media_errors[72h]) > 200
for: 5m
labels:
severity: warning
annotations:
summary: 'Megaraid Physical Disk media error count increased by
{{$value | humanize}} over 72h'
Every time the expr is evaluated, it's looking over the most recent 72
hours. "increase" is like "rate", but its output is scaled up to the time
period in question - i.e. instead of rate per second, it gives rate per 72
hours in this case.
> i tried to use the promql function rate but in this case my first
occurence is missing.
"rate" (and "increase") calculate the rate between two data points. If the
timeseries has only one data point, it cannot give a result. It cannot
assume that the previous data point was zero, because in general that may
not be the case: prometheus could have been started when the counter was
already above zero.
You should make your timeseries spring into existence with value 0 at the
start.
On Wednesday, 22 June 2022 at 09:27:52 UTC+1 Loïc wrote:
> Hi,
>
> I use an exporter mtail to alerting when a pattern match into the
> kubernetes logs. When my alarm is firing, i would like auto-resolved it. I
> search how to use tje endsat parameter in my rule but i don't found.
>
> Also, i tried to use the promql function rate but in this case my first
> occurence is missing.
>
> Have you an idea ?
>
> Thanks
> Loïc
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/d6d8ee04-41e9-4c78-98a1-9fb5a4278742n%40googlegroups.com.
