I suspect this issue is generated when alerts are in bulk. Suppose I have
applied the rule to alert me if interfaces are down on 4 to 5 devices. And
these alerts are 40 to 50 in number then alertmanager is not able to
forward those alerts to webhook.
Later on I changed alert manager YAML file to:
*global:
resolve_timeout: 1d
route:
receiver: netorc_webhook
group_by: ['...']
group_wait: 0s
group_interval: 10s
receivers:
- name: "netorc_webhook"
webhook_configs:
- url: http://nginx:8082/v1/alerts/alert-manager*
I was still facing the issue. I then added repeat_nterval to the 30s. But
now I start to get
*level=warn ts=2020-02-15T05:43:17.019Z caller=delegate.go:272
component=cluster msg="dropping messages because too many are queued"
current=4118 limit=4096*
On Thursday, February 13, 2020 at 7:40:29 PM UTC+5, Simon Pasquier wrote:
>
> By default Alertmanager will wait 30s before sending the notification
> (that's the group_wait parameter).
> Can you be more specific about the issue you're seeing?
>
> On Tue, Feb 11, 2020 at 5:54 AM Kiran Ali <[email protected]
> <javascript:>> wrote:
> >
> > I'm using alertmanager v0.19.0 and my alertmanager configs are:
> >
> > global:
> > resolve_timeout: 1d
> > route:
> > receiver: webhook
> > receivers:
> > - name: "webhook"
> > webhook_configs:
> > - url: http://ip/v1/alerts
> > send_resolved: true
> >
> > Alerts are generated on time from Prometheus and forwarded to
> alertmaanger. As soon as alertmanager receives alerts it should forward it
> to webhook but there is no proper time when alertmaanger will forward
> alerts to webhook. I was facing same issue with old alertmaanger version as
> well.
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Prometheus Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to [email protected] <javascript:>.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/5896ebd5-4a86-4a3e-845d-dacaa5123122%40googlegroups.com.
>
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/8fcfab27-5948-44d9-873f-1f9485003309%40googlegroups.com.
