Hi everyone, The Alertmanager Time Interval design doc is now in its final stages. Again I want to thank you all for feedback and comments, they have been very productive and helpful.
The final design allows users to specify a hierarchy of receivers with valid times for a route, and Alertmanager will choose the first receiver that matches the time of the alert firing. I have provided some examples in the design doc of how this might be used. I will leave the document open for the rest of the week for final comments and review, and then I will close it for comments and hopefully we can begin implementation. You can find the document here <https://docs.google.com/document/d/1pf-rPDQUGJUHazyr5vanTO6ft3loNZO9UoVpvhShFtA/edit?usp=sharing> . Cheers, Ben On Tue, Jun 16, 2020 at 11:55 AM Benjamin Ridley <[email protected]> wrote: > Hi everyone, > > Thank you all for your feedback on the Alertmanager Time Interval design > doc so far. The design has been greatly simplified and (in my opinion) > improved already due to the feedback received. > > I've made some changes to the proposed implementation so that it works on > receivers now, not the routes themselves. Brian pointed out that this > approach requires users to define many of their routes twice, with the only > difference being the active time interval and which receiver to use. For > example, notice how in the below snippet the 'severity: warning' alerts > have two routes but really all that needs to change is the receiver and > time: > >> - match: >> severity: warning >> time_intervals: >> include: >> - business_hours >> exclude: >> - public_holidays >> receiver: team-X-pager >> continue: true >> - match: >> severity: warning >> receiver: team-X-slack >> time_intervals: >> exclude: >> - business_hours > > > So the current proposal is to introduce a 'timed_receivers' section of a > route that pairs receivers and time intervals. This way the above route is > simplified into a single block: > >> - match: >> severity: warning >> receiver: team-X-slack >> timed_receivers: >> - receiver: team-X-pager >> include_intervals: >> - business_hours >> exclude_intervals: >> - public_holidays >> > > Additionally, this approach maintains the desirable characteristics of > previous solutions in that it has no impact on existing routing decisions. > Adding the tag also maintains backwards compatibility for existing > configurations. > > Please let me know what you think, either here or in the comments of the > design doc found here. > <https://docs.google.com/document/d/1pf-rPDQUGJUHazyr5vanTO6ft3loNZO9UoVpvhShFtA/edit?usp=sharing> > > Cheers, > Ben > > On Mon, Jun 8, 2020 at 1:31 PM Benjamin Ridley <[email protected]> > wrote: > >> Hi everyone, >> >> I'm sure many of you have come across the problem of controlling alerts >> based on the time of day or outside business hours etc inside Alertmanager. >> There is also a longstanding issue >> <https://github.com/prometheus/alertmanager/issues/876> on the >> Alertmanager GitHub about this which I encourage you to read if you want >> some more context. >> >> This is a proposed design for defining time intervals in the Alertmanager >> configuration file and how they would be used in the routing tree to >> silence particular routes inside or outside the specified intervals, >> allowing users to model time-based requirements to their liking. >> >> The document is open for suggestions and comments and any feedback is >> welcomed, so please take a look and let us know what you think. You can >> access the document here >> <https://docs.google.com/document/d/1pf-rPDQUGJUHazyr5vanTO6ft3loNZO9UoVpvhShFtA/edit?usp=sharing> >> . >> >> Cheers, >> Ben >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/aa340edb-eb89-4dd8-af60-886d0427d3c7o%40googlegroups.com >> <https://groups.google.com/d/msgid/prometheus-developers/aa340edb-eb89-4dd8-af60-886d0427d3c7o%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CABaapwJ-%2BQiDSjxTLzJguL6YbtnS0POVs_7zRPWVBCVERBtnSw%40mail.gmail.com.
