1. Are you sure that the alerts you generate from alerting rules have the
'severity: critical' or 'severity: warning' labels on them? If not, they
won't match any of the routes, so they'll fall back to the default you set:
route:
receiver: slack_general
2. Why do you have these empty routes?
- match:
severity: critical
continue: true
- match:
severity: warning
continue: true
They don't do anything - delete them.
3. In order to see if alertmanager is attempting to send to opsgenie (and
failing):
* Look at the logs of the alertmanager proces (e.g. "journalctl -eu
alertmanager" if running it under systemd)
* Look at the notification metrics which alertmanager itself generates:
curl -Ss localhost:9093/metrics | grep
'alertmanager_notifications.*opsgenie'
If you see:
alertmanager_notifications_failed_total{integration="opsgenie"} 0
alertmanager_notifications_total{integration="opsgenie"} 0
then no delivery to opsgenie has been attempted. If there are attempts and
failures, you'll see these metrics going up.
BTW, it's useful to scrape alertmanager from your prometheus, so you can
query these metrics and get history of them (and indeed, alert on them if
necessary):
- job_name: alertmanager
scrape_interval: 1m
static_configs:
- targets: ['localhost:9093']
4. If you want to deliver a particular alert to multiple destinations, a
much cleaner way of doing it is to use a subtree of routes to list multiple
destinations:
- match:
severity: critical
routes: [ {receiver: slack_general, continue: true}, {receiver:
netmera_opsgenie} ]
Then you don't have to duplicate your matching logic (in this case
"severity: critical"), and you don't get into confusion over when to use
"continue: true".
OTOH, if you want *all* alerts to go to slack regardless, then just put a
catch-all route at the top:
route:
receiver: slack_general # this is never used because the first rule
below always matches
routes:
* - receiver: slack_general continue: true*
- match:
severity: critical
receiver: 'netmera_opsgenie'
- match:
severity: warning
receiver: 'netmera_opsgenie'
5. "match" and "match_re" are deprecated, better to start using the new
matchers syntax:
- matchers:
- 'severity =~ "warning|critical"'
receiver: 'netmera_opsgenie'
On Monday, 17 January 2022 at 12:55:49 UTC Hasene Ceren Yıkılmaz wrote:
> When I restart the alertmanager it's running, but I can't see any of these
> alerts in OpsGenie.
> I follow this doc : https://support.atlassian.com/opsgenie/docs/integrate-
> opsgenie-with-prometheus/ and this doc
> https://prometheus.io/docs/alerting/latest/configuration/#opsgenie_config
> Is there anything to controlling about Alertmanager & OpsGenie
> integration?
>
> This is my alertmanager.yml file;
>
> global:
> resolve_timeout: 5m
>
> route:
> receiver: slack_general
> group_by: ['instance']
> group_wait: 1s
> group_interval: 1s
> routes:
> - match:
> severity: critical
> continue: true
> receiver: slack_general
> - match:
> severity: warning
> continue: true
> receiver: slack_general
> - match:
> severity: critical
> continue: true
> - match:
> severity: warning
> continue: true
> # added receivers for opsgenie
> - match:
> severity: critical
> receiver: 'netmera_opsgenie'
> - match:
> severity: warning
> receiver: 'netmera_opsgenie'
>
> receivers:
> - name: slack_general
> slack_configs:
> - api_url: 'slack api url'
> channel: '#netmera-prometheus'
> send_resolved: true
> title: "{{ range .Alerts }}{{ .Annotations.summary }}\n{{ end }}"
> text: "{{ range .Alerts }}{{ .Annotations.description }}\n{{ end }}"
> # added opsgenie configs
> - name: 'netmera_opsgenie'
> opsgenie_configs:
> - api_key: opsgenie api key
> api_url: https://api.eu.opsgenie.com/
> message: '{{ range .Alerts }}{{ .Annotations.summary }}\n{{ end }}'
> description: '{{ range .Alerts }}{{ .Annotations.description }}\n{{
> end }}'
> priority: '{{ range .Alerts }}{{ if eq .Labels.severity
> "critical"}}P2{{else}}P3{{end}}{{end}}'
>
> I contacted with OpsGenie support and they checked the logs, but they
> couldn't see anything comes from alert manager.
> Could you please help me about that?
>
> Thank you!
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/16679603-4cfb-4e25-bfc2-315a3dbc9b5cn%40googlegroups.com.
