pvary commented on code in PR #9308:
URL: https://github.com/apache/iceberg/pull/9308#discussion_r1430516299
##########
flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergSourceSplitReader.java:
##########
@@ -123,6 +155,19 @@ public void close() throws Exception {
}
}
+ @Override
+ public void pauseOrResumeSplits(
+ Collection<IcebergSourceSplit> splitsToPause,
Collection<IcebergSourceSplit> splitsToResume) {
+ LOG.info("Pause splits: {} and resume splits: {}", splitsToPause,
splitsToResume);
+ pausedSplits.addAll(
Review Comment:
If the SplitFetcher realizes that the reader should be paused, then it will:
- Calls the `wakeUp` to unblock the current `fetch`
- When it gets back the control (after the fetch is unblocked), then it
calls `pauseOrResumeSplits`
- After `pauseOrResumeSplits` it calls the fetch again
The SplitFetcher keeps a `taskQueue` which contains the changes on the
splits. If there is no task, then if goes to `fetch` and if there is nothing in
the queue and the fetch result is empty, then the fetcher is closed. The only
method which gets called in parallel is the `wakeUp` to break the `fetch` loop
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
