Hi,
I am observing an issue with AlertManager recently. I am running alertmanager as a pod and it works fine. Alerts from Prometheus are reaching and it's triggering alerts. It's silencing alerts created from the UI and curl command. The only issue is - suppose I am creating a silence for a particular host, which has multiple servers, the silencing works fine but 1 or 2 alerts still get spilled into the notification channel. I want to understand why this would happen. Either silence will not work at all for all servers belonging to the host, or silence would work for all the servers for that host. Why is this spilling alerts for any one or two servers for the particular host ? Is it because too many alerts were getting generated and it was breaching a specified queue size for silenced alerts? Please help me understand under what circumstance alertmanager would spill a few alerts even though the alerts were all silenced. Please note : only a few alerts are notifying despite the silence - sometimes one, sometimes 2, at random. Thanks, Arnav -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/47562231-8166-4d51-b257-2326921392b4n%40googlegroups.com.
