Hi. I have around 80 alert rules configured. These days I have started receiving warning from Alertmanager saying "*component=cluster msg="dropping messages because too many are queued" current=4130 limit=4096* ". What exactly does this mean? Are 4130 alerts in the pending queue? How can so many alerts generate? I have currently 430 alerts(Warning + Critical). How are these many alerts queued? And what to do to overcome this issue?
Below are my global alertmanager configurations. global: resolve_timeout: 5m slack_api_url: "blahblah" route: group_by: ['cluster'] group_wait: 10s group_interval: 10s repeat_interval: 1h receiver: 'slack-channel' Also, out of 80 alert rules, some of them are for warning, for which I don't need an alert notification but I need them for my visualization and dash-boarding purposes. Is there any way to disable some of the alert-rules to not even go to the global receiver? Can someone please help here? Some of my critical alerts are getting blocked. Thanks in advance. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/01ccc40b-4c20-4140-8641-b6984da370f8n%40googlegroups.com.
