haishui126 commented on PR #25678: URL: https://github.com/apache/flink/pull/25678#issuecomment-2493319123
My understanding of the watermark idle: The SourceOperator will create 1+n outputs (MainOutput * 1 and PerSplitOutput * n), and when mainOutput and all PerSplitOutputs are idle, the downstream will receive IDLE watermarkStatus. Some data sources use MainOutput, such as DataGen, after the program starts the idleTimeout time, PerSplitOutput will automatically become idle. The same as the PerSplitOutput used by Kafka, so MainOutput will automatically become idle, when the watermark is aligned, We're going to stop the PausableRelativeClock for MainOutput and PerSplitOutput By the way, if the SourceReader is using MainOutput, since SourceOperator#updateCurrentSplitWatermark will not be called, the splitCurrentWatermarks will be empty. So don't worry about the source will try to stop the PerSplitOutput's Clock. Please consider if there are any effects that I haven't taken into account when calling pauseOrResumeSplits with only one split present. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
