It looks like a good idea.But I'm using alertManager to receive all the
alerts, including calling the alertManager API directly to create the
alerts, and I can't forbid the client not to call for a certain period of
time, so I want to set it on alertManager to be silent
在2020年11月24日星期二 UTC+8 上午5:24:35<Christian Hoffmann> 写道:
> Hi,
>
> On 11/21/20 2:41 PM, Aleksandar Ilic wrote:
> > I was wondering if there is any way to set an alert only to be triggered
> > at a specific time of day.
> >
> > As I saw for alertmanager there is PR open on GitHub but wondering if
> > there is any workaround for this or any other way.
>
> What David suggested would be the simplest method, I think. It has a
> drawback though: The alert will resolve once the working hours pass.
>
> We use the following pattern instead:
>
> * Create a pseudo-alert InhibitOutOfWorkingHours which fires except
> during the relevant working hours. We want this to be localtime-aware,
> which is why we don't use hour() but a textfile collector-provided
> metric called localtime_hour, etc.
>
> * We use alert relabeling to automatically add the label
> inhibited_by=<alertname> to all Inhibit.* alerts (i.e.
>
> {alertname="InhibitOutOfWorkingHours",inhibited_by="InhibitOutOfWorkingHours"})
>
> * We set up a generic inhibition rule which takes source alerts with
> alertname="Inhibit.+" and inhibits target alerts with equal inhibited_by
> labels.
>
> * The alert(s) which should be inhibited during certain timeframes are
> modified to have the appropriate label (e.g.
> inhibited_by="InhibitOutOfWorkingHours")
>
>
> Might sound complicated, but it works fine so far and is supported by
> our configuration management logic.
>
> Kind regards,
> Christian
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/91967dd5-dea9-4596-9170-7d20428eb1a8n%40googlegroups.com.