> Does anyone know how to set the time between firing and resolved
> alert?
>
> I thought it is resolve_timeout after "global" line in
> alertmanager.yml, but it not working. Is there something that need to
> be configured additionally?
The resolve_timeout setting in Alertmanager is in practice unused
in modern Prometheus setups, and there should probably be a comment
to that effect in the examples. (It keeps misleading people.)
In Prometheus, alerts are resolved immediately when their alert
expression is no longer true. If you need the alert to continue to be
active in Prometheus for some minimum time, you'll need to figure out
an alert expression that guarantees this. For example, instead of using
'something > 100', you might use 'max_over_time(something[5m]) > 100'
to insure that it stays true for at least five minutes after it first
happens.
In Alertmanager, notifications about resolved alerts are handled the
same as all other changes in an alert group; you get notified about
all of the recent changes only once every group_interval time period,
starting from when the group first became active. This means that if
you set group_interval to five minutes, I believe that you won't get
any notifications about resolved alerts until at least five minutes
after the first alert fired. However, this doesn't create any minimum
delay between an alert being resolved and you being notified about
it. If Alertmanager started the alert at 10:00:00 with a five minute
group_interval and the alert is resolved at 10:04:58, Alertmanager will
tell you about it at 10:05:00 because that's five minutes since the last
notification. But if the alert is resolved at 10:05:01, you'll only hear
about it at 10:10:00, at the next group_interval tick.
- cks
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/20210520191043.D400F321C44%40apps1.cs.toronto.edu.
