On Thu, 18 Jun 2020 at 10:08, Sayan Das <[email protected]> wrote: > Hi, > I have multiple prometheus instances sending alerts to one alertmanager, > which deduplicates the alerts. To ensure alerting pipeline between each > prometheus and alertmanager is working properly, I want to send a always > active alert to the alertmanager. I am configuring a alert rule in each > prometheus which would always trigger. I have rule: > > - alert: WatchDog > > annotations: > > message: 'This alert should always be active in alertmanager, to > ensure the prometheus-alertmanager integration is functional. We should be > notified immediately if this alert is not active' > > expr: vector(1) > > labels: > > severity: none > > The tricky part is how do we differentiate this alert in alertmanager, > since all prometheuses send the same alert. Is there a way to add labels to > a vector v before sending it to the alert manager. I saw prometheus has > alert_relabel_configs, but how do we use it to add labels to a vector v has > no labels ? Can I somehow use label_replace() or label_join() functions for > this use case? >
Each Prometheus should have distinct external_labels, which will be added to all alerts to indicate they're from. -- Brian Brazil www.robustperception.io -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAHJKeLrVt5bSz9PP%3Dyb180Vez3Ds5gqXxhJKOToHBOtpPfYxDw%40mail.gmail.com.
