What Chris says: Prometheus will send a resolved alert to Alertmanager. If Alertmanager didn't receive any firing alert for the group, it won't send the resolved notification. You may get more insights by running Alertmanager with --log.level=debug.
On Tue, Feb 18, 2020 at 8:07 PM Chris Siebenmann <[email protected]> wrote: > > > If I understand correctly, prometheus doesn't send any "resolved" > > message to alertmanager: it just stops sending alerts. Alertmanager > > treats the lack of alert as meaning "resolved". > > > > Therefore, if you receive the "resolved" message, then this proves > > that alertmanager must have received the original alert. > > > > Hence you need to check the alertmanager logs to determine why it > > couldn't send out the original alert E-mail. > > Modern versions of Prometheus do send specific 'resolved' messages to > Alertmanager when an alert ends. I don't know what Alertmanager does > if it receives a 'resolved' message for an alert that it hasn't seen > before, though. > > If Prometheus stops periodically sending information about current > alerts to Alertmanager (for example because it's been stopped or has > crashed), the current alerts will normally time out in a few minutes and > be declared as resolved by Alertmanager. But normally this shouldn't > happen; Prometheus re-notifies Alertmanager about any current alerts > on a regular basis. > > - cks > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/20200218190718.AAC183201A1%40apps1.cs.toronto.edu. > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAM6RFu5S5OtU%2B1jKvHMrM9-0Fgb3E0BkF3CGGcgtVt8%3D6S6GwA%40mail.gmail.com.
