Firstly, I'd drop the "continue: true" lines. They are not required, and
are just going to cause confusion.
The 'slack' and 'production' receivers are both sending to #prod-channel.
So you'll hit this if the env is not exactly "dev". I suggest you look in
detail at the alerts themselves: maybe they're tagging with "Dev" or "dev "
(with a hidden space).
If you change the default 'slack' receiver to go to a different channel, or
use a different title/text template, it will be easier to see if this is
the problem or not.
On Friday, 12 August 2022 at 09:36:22 UTC+1 rs wrote:
> Hi everyone! I am configuring alertmanager to send outputs to a prod slack
> channel and dev slack channel. I have checked with the routing tree editor
> and everything should be working correctly.
> However, I am seeing some (not all) alerts that are tagged with 'env: dev'
> being sent to the prod slack channel. Is there some sort of old
> configuration caching happening? Is there a way to flush this out?
>
> --- Alertmanager.yml ---
> global:
> http_config:
> proxy_url: 'xyz'
> templates:
> - templates/*.tmpl
> route:
> group_by: [cluster,alertname]
> group_wait: 10s
> group_interval: 30m
> repeat_interval: 24h
> receiver: 'slack'
> routes:
> - receiver: 'production'
> match:
> env: 'prod'
> continue: true
> - receiver: 'staging'
> match:
> env: 'dev'
> continue: true
> receivers:
> #Fallback option - Default set to production server
> - name: 'slack'
> slack_configs:
> - api_url: 'api url'
> channel: '#prod-channel'
> send_resolved: true
> color: '{{ template "slack.color" . }}'
> title: '{{ template "slack.title" . }}'
> text: '{{ template "slack.text" . }}'
> actions:
> - type: button
> text: 'Query :mag:'
> url: '{{ (index .Alerts 0).GeneratorURL }}'
> - type: button
> text: 'Silence :no_bell:'
> url: '{{ template "__alert_silence_link" . }}'
> - type: button
> text: 'Dashboard :grafana:'
> url: '{{ (index .Alerts 0).Annotations.dashboard }}'
> - name: 'staging'
> slack_configs:
> - api_url: 'api url'
> channel: '#staging-channel'
> send_resolved: true
> color: '{{ template "slack.color" . }}'
> title: '{{ template "slack.title" . }}'
> text: '{{ template "slack.text" . }}'
> actions:
> - type: button
> text: 'Query :mag:'
> url: '{{ (index .Alerts 0).GeneratorURL }}'
> - type: button
> text: 'Silence :no_bell:'
> url: '{{ template "__alert_silence_link" . }}'
> - type: button
> text: 'Dashboard :grafana:'
> url: '{{ (index .Alerts 0).Annotations.dashboard }}'
> - name: 'production'
> slack_configs:
> - api_url: 'api url'
> channel: '#prod-channel'
> send_resolved: true
> color: '{{ template "slack.color" . }}'
> title: '{{ template "slack.title" . }}'
> text: '{{ template "slack.text" . }}'
> actions:
> - type: button
> text: 'Query :mag:'
> url: '{{ (index .Alerts 0).GeneratorURL }}'
> - type: button
> text: 'Silence :no_bell:'
> url: '{{ template "__alert_silence_link" . }}'
> - type: button
> text: 'Dashboard :grafana:'
> url: '{{ (index .Alerts 0).Annotations.dashboard }}'
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/a6cf306e-2dee-4c07-ba64-ae439ce96182n%40googlegroups.com.
