Re: Counting elements that appear "behind" the watermark

Awesome, thanks Elias! On Tue, Jul 31, 2018 at 10:02 PM, Elias Levy wrote: > Correct. Context gives you access to the element timestamp > . > But it

Re: Counting elements that appear "behind" the watermark

Correct. Context gives you access to the element timestamp . But it also gives you access to the current watermark via timerService

Re: Counting elements that appear "behind" the watermark

You can create a ProcessFunction. That gives you access to getRuntimeContext to register metrics, to the element timestamp, and the current watermark. Keep in mind that operators first process a record and then process any watermark that was the result of that record, so that when you get the

Re: Counting elements that appear "behind" the watermark

Hi Julio, If I understand correctly, you want to adjust your watermarks automatically? It is true that there are no direct ways to get metric from the AssignerWithPeriodicWatermarks. Adding ProcessFunction before assignTimestampsAndWatermarks seems a solution. In the ProcessFunction, you can

Counting elements that appear "behind" the watermark

Hello, Our current watermark model is "some time behind the most recent seen element" (very close to what the docs have in "Periodic Watermark" https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/event_timestamps_watermarks.html#with-periodic-watermarks). It fits our current