I suspect this issue is generated when alerts are in bulk. Suppose I have applied rule: Alert me when interfaces are down on 4 to 5 devices. And these alerts are 40 to 50 in number then alertmanager is not able to forward those alerts to webhook.
Later on I changed alert manager YAML file to: *global: resolve_timeout: 1d route: receiver: webhook group_by: ['...'] group_wait: 0s group_interval: 10s receivers: - name: "webhook" webhook_configs: - url: http://nginx:8082/v1/alerts <http://nginx:8082/v1/alerts/alert-manager>* I was still facing the issue. I then added repeat_nterval to the 30s. But now I start to get *level=warn ts=2020-02-15T05:43:17.019Z caller=delegate.go:272 component=cluster msg="dropping messages because too many are queued" current=4118 limit=4096* On Thursday, February 13, 2020 at 7:40:29 PM UTC+5, Simon Pasquier wrote: > > By default Alertmanager will wait 30s before sending the notification > (that's the group_wait parameter). > Can you be more specific about the issue you're seeing? > > On Tue, Feb 11, 2020 at 5:54 AM Kiran Ali <[email protected] > <javascript:>> wrote: > > > > I'm using alertmanager v0.19.0 and my alertmanager configs are: > > > > global: > > resolve_timeout: 1d > > route: > > receiver: webhook > > receivers: > > - name: "webhook" > > webhook_configs: > > - url: http://ip/v1/alerts > > send_resolved: true > > > > Alerts are generated on time from Prometheus and forwarded to > alertmaanger. As soon as alertmanager receives alerts it should forward it > to webhook but there is no proper time when alertmaanger will forward > alerts to webhook. I was facing same issue with old alertmaanger version as > well. > > > > -- > > You received this message because you are subscribed to the Google > Groups "Prometheus Users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected] <javascript:>. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/5896ebd5-4a86-4a3e-845d-dacaa5123122%40googlegroups.com. > > > > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/5936af61-a4bd-4028-92f6-b57fe9d99e53%40googlegroups.com.
