We're using two groups of Alerts
- common for all teams
- per-team only
In prometheus config it looks like this:
rule_files:
- /git-sync/common/*.yaml
- /git-sync/rules/*.yaml
And directory git-sync comes from team repo. In this repo folder /common/
comes from git submodule.
This mostly works fine, but there are corner cases when team want to
override some common Alert rule on their level.
Currently it is not possible, consider this case:
# /git-sync/common/prom.yaml
groups:
- name: prom
rules:
- alert: InstanceDown
expr: up == 0
for: 5m
labels:
severity: critical
annotations:
summary: Instance scrape failed
# /git-sync/rules/prom.yaml
groups:
- name: prom
rules:
- alert: InstanceDown
expr: up{group!="maintenance"} == 0
for: 5m
labels:
severity: critical
annotations:
summary: Instance scrape failed
This still loads as 2 different alerts in Prometheus, and first one
continues to fire.
Is it possible to override alerts?
Shouldn't alert names be unique?
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/d4532c99-835e-4c0c-b145-9e662a965f2f%40googlegroups.com.
