kbendick commented on code in PR #4943:
URL: https://github.com/apache/iceberg/pull/4943#discussion_r889650706
##########
flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/source/StreamingMonitorFunction.java:
##########
@@ -155,6 +162,12 @@ private void monitorAndForwardSplits() {
if (lastSnapshotId == INIT_LAST_SNAPSHOT_ID) {
newScanContext = scanContext.copyWithSnapshotId(snapshotId);
} else {
+ List<Long> snapshotIds = SnapshotUtil.snapshotIdsBetween(table,
lastSnapshotId, snapshot.snapshotId());
+ if (snapshotIds.size() < scanContext.monitorSnapshotNumber()) {
+ snapshotId = snapshot.snapshotId();
+ } else {
+ snapshotId = snapshotIds.get(snapshotIds.size() -
scanContext.monitorSnapshotNumber());
+ }
Review Comment:
I’m thinking through these situations now.
But users would _never enter this block_ if they don’t opt into the new
behavior, is that correct?
If so, can we add a conditional case that this block won’t be entered unless
the user have a non `INT_MAX` value (eg we make a Precondition and just make a
sentence that starts with `[bug] this shouldn’t happen <because>`). We have one
or two other places that use the same `[bug]` syntax and this new logic change
would ideally not apply to users that keep the default behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
