cc user@, bcc dev@ Hi Dhruv,
Yes, this should be totally possible. For 1, I would use a ProcessFunction to buffer the alerts and register timers per alert for the repeated firing (or to clear it from state if it is resolved). From this ProcessFunction you send records to an AlertManagerSink. For 2. the AsyncSink, Fabian Paul (cc) might be the best person to judge if it is a good fit. There is PR [1] to add a blog post about the Async Sink that might be of help for you in the meantime. Cheers, Konstantin [1] https://github.com/apache/flink-web/pull/517 On Thu, Apr 14, 2022 at 9:43 AM Dhruv Patel <dhruvpatel5...@gmail.com> wrote: > Hi, > We have a use case where we want to send alerts to Prometheus > Alertmanager (https://prometheus.io/docs/alerting/latest/alertmanager/) > from Flink. Each of these alerts have a startsAt, endsAt field along with > alertMetadata. Alertmanager expects clients to send all the FIRING alerts > every X minutes (X is configurable) with an updated endsAt time (greater > than X since alertmanager would drop the alert once endsAt time is reached) > and once the alert is in RESOLVED state stop sending it. The state updates > of the alerts would come from Kafka. So the pipeline is something like this > State Updates to Alerts (Kafka) -> Flink (Some Enrichment) -> Alertmanager > Sink > They provide a rest endpoint where we can post these alerts. I have some > questions to see if its achievable to develop a sink connector for > alertmanager in flink? > > 1. Is it possible to send data to a sink every X minutes from a custom sink > connector since I have to somehow simulate a behavior of continuously > sending the same alerts even because state updates are only received from > Kafka for FIRING -> RESOLVED state and not for FIRING -> FIRING state? I > was thinking of having a state of active alerts and somehow the sink > connector would get the state every X minutes, batch it and then send it to > alertmanager. However, I am not able to find resources to write some > implementation around it. > > 2. In Flink 1.15, there are Async Sinks ( > https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink) > but not much documentation around. Also don't know if it would be > achievable to write the continuous firing logic in alertmanager > > Any other ideas are welcomed. > > -- > *Thanks,* > *Dhruv* > -- Konstantin Knauf https://twitter.com/snntrable https://github.com/knaufk
