Hi,

Let's consider two operators: A (parallelism=2) and B (parallelism=1).
B has two input partitions, B_A1 and B_A2, which are connected to A1 and A2
respectively.

At some point,
- B_A1's watermark : 12
- B_A2's watermark : 10
- B's event-time clock : 10 = min(12, 10)
- B has registered a timer at 12
- No data will be fed into the pipeline for the next few hours, but I want
the timer to be fired after a few seconds if no data is coming.

After adopting a watermark strategy explained in [1], I found that the
timer is fired as wished! That's awesome!

But I want to know what happens inside in detail.
Based on my current understanding of how watermark is calculated [2], I
cannot come up with what happens inside when idleness is considered.
If B_A2 is marked idle earlier than B_A1, is B's event-time clock
calculated as min(12, MAX_WATERMARK)?

Thanks,

Dongwon

[1]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_timestamps_watermarks.html#dealing-with-idle-sources
[2]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_timestamps_watermarks.html#how-operators-process-watermarks
[3]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-126%3A+Unify+%28and+separate%29+Watermark+Assigners

Reply via email to