Re: monitoring message latency for flink sql app

2024-05-16 Thread Hang Ruan
Hi, mete. As Feng Jin said, I think you could make use of the metric ` currentEmitEventTimeLag`. Besides that, if you develop your job with the DataStream API, you could add a new operator to handle it by yourself. Best, Hang Feng Jin 于2024年5月17日周五 02:44写道: > Hi Mete > > You can refer to the

Re: monitoring message latency for flink sql app

2024-05-16 Thread Feng Jin
Hi Mete You can refer to the metrics provided by the Kafka source connector. https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/connectors/datastream/kafka//#monitoring Best, Feng On Thu, May 16, 2024 at 7:55 PM mete wrote: > Hello, > > For an sql application using kafka as

RE: monitoring message latency for flink sql app

2024-05-16 Thread Anton Sidorov
Hello mete. I found this SO article https://stackoverflow.com/questions/54293808/measuring-event-time-latency-with-flink-cep If I'm not mistake, you can use Flink metrics system for operators and get time of processing event in operator. On 2024/05/16 11:54:44 mete wrote: > Hello, > > For an

monitoring message latency for flink sql app

2024-05-16 Thread mete
Hello, For an sql application using kafka as source (and kafka as sink) what would be the recommended way to monitor for processing delay? For example, i want to be able to alert if the app has a certain delay compared to some event time field in the message. Best, Mete