shanzi commented on PR #16545: URL: https://github.com/apache/iceberg/pull/16545#issuecomment-4585921082
@pvary Studied (with the help of claude) the related code, I think the refactored code should work correctly now. The problem is previously the `pauseOrResumeSplits` and `wakeUp` method is left empty for no-op (see this [comment](https://github.com/apache/iceberg/pull/16545#discussion_r3328225775)), so to currently wakeUp during shutdown, we need to make sure it doesn't affect watermark alignement. However, there is no way to distinguish if wakeUp is called for watermark alignment or shutdown in Flink so far. We need to make sure it returns an emptyBatch and let it go back to the `SourceFetcher`. If it is watermark alignment, the fetcher may resume the fetch again. Otherwise, the loop will exits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
