leonardBang commented on code in PR #3548:
URL: https://github.com/apache/fluss/pull/3548#discussion_r3499778754
##########
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/source/enumerator/FlinkSourceEnumerator.java:
##########
@@ -978,6 +1100,12 @@ private void handleSplitsAdd(List<SourceSplitBase>
splits, Throwable t) {
t);
}
}
+
+ initialDiscoveryFinished = true;
+ for (SourceSplitBase split : splits) {
+ unassignedSplits.put(split.getTableBucket(), split);
Review Comment:
This stores unassigned splits by TableBucket, but TableBucket is not unique
for all split types. LakeSnapshotSplit uses splitIndex to distinguish multiple
lake splits in the same bucket, so checkpointing only unassignedSplits.values()
can drop all but one split (or fail on restore with duplicate keys). Could we
persist these by splitId or as a list instead of de-duplicating by TableBucket?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
