Thanks Hequn for the pointer. From what I read, I may also need to emit the timestamp regularly for all idle partitions to ensure watermark progression.
— Fritz > On Nov 8, 2018, at 6:02 PM, Hequn Cheng <chenghe...@gmail.com> wrote: > > Hi Fritz, > > Watermarks are merged on stream shuffles. If one of the input's watermark not > progressing, they will not advance the event time at the operators. I think > you should decrease the parallelism of source and make sure there are data in > each of your source partition. > Note that the Kafka source supports per-partition watermarking, which you can > read more about here[1]. > > Best, Hequn > [1] > https://ci.apache.org/projects/flink/flink-docs-master/dev/event_timestamps_watermarks.html#timestamps-per-kafka-partition > > <https://ci.apache.org/projects/flink/flink-docs-master/dev/event_timestamps_watermarks.html#timestamps-per-kafka-partition> > > > On Fri, Nov 9, 2018 at 1:56 AM Fritz Budiyanto <fbudi...@icloud.com > <mailto:fbudi...@icloud.com>> wrote: > Hi All, > > I noticed if one of the slot's watermark not progressing, its impacting all > slots processFunction timer and no timer are not firing. > > In my example, I have Source parallelism set to 8 and Kafka partition is 4. > The next operator is processFunction with parallelism of 8 + event timer. I > can see from the debug log that one of the slot's watermark is not > progressing. As a result, all slot's timer in the process function are not > firing. Is this expected behavior or issue? How do I prevent this condition? > > Thanks, > Fritz
