u70b3 commented on code in PR #9037:
URL: https://github.com/apache/paimon/pull/9037#discussion_r3728341621
##########
paimon-core/src/main/java/org/apache/paimon/utils/SnapshotManager.java:
##########
@@ -361,9 +361,13 @@ public boolean earliestFileNotExists() {
public @Nullable Snapshot earlierOrEqualWatermark(long watermark) {
Long latest = latestSnapshotId();
+ if (latest == null) {
+ return null;
+ }
// If latest == Long.MIN_VALUE don't need next binary search for
watermark
// which can reduce IO cost with snapshot
- if (latest == null || snapshot(latest).watermark() == Long.MIN_VALUE) {
+ Long latestWatermark = snapshot(latest).watermark();
+ if (latestWatermark != null && latestWatermark == Long.MIN_VALUE) {
Review Comment:
One note on the removed fast path: it existed to save snapshot IO for batch
tables where every watermark is the sentinel. With the uniform
missing-handling, that case now scans retained snapshots until the first real
watermark appears (or the scan exhausts) — bounded by snapshot.num-retained and
amortized by the snapshot cache in SnapshotManager. We kept no early return
because there is no sound one: proving "all snapshots lack a real watermark"
requires inspecting all of them, and any check based only on the latest
snapshot misjudges mixed histories like [100, Long.MIN_VALUE].
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
