Re: Counting elements that appear "behind" the watermark

2018-08-01 Thread Julio Biason
Awesome, thanks Elias! On Tue, Jul 31, 2018 at 10:02 PM, Elias Levy wrote: > Correct. Context gives you access to the element timestamp > . > But it

Re: Counting elements that appear "behind" the watermark

2018-07-31 Thread Elias Levy
Correct. Context gives you access to the element timestamp . But it also gives you access to the current watermark via timerService

Re: Counting elements that appear "behind" the watermark

2018-07-30 Thread Elias Levy
You can create a ProcessFunction. That gives you access to getRuntimeContext to register metrics, to the element timestamp, and the current watermark. Keep in mind that operators first process a record and then process any watermark that was the result of that record, so that when you get the

Re: Counting elements that appear "behind" the watermark

2018-07-30 Thread Hequn Cheng
Hi Julio, If I understand correctly, you want to adjust your watermarks automatically? It is true that there are no direct ways to get metric from the AssignerWithPeriodicWatermarks. Adding ProcessFunction before assignTimestampsAndWatermarks seems a solution. In the ProcessFunction, you can