u70b3 commented on PR #677: URL: https://github.com/apache/paimon-rust/pull/677#issuecomment-5205198000
Thanks for checking this case — I can confirm the reproduction: for `[100, Long.MIN_VALUE]` querying `100`, this implementation returns snapshot 1 while released Java returns `null`. This is the deliberate difference documented in the PR description, so let me explain the reasoning and how I'd propose to sequence it. The Rust behavior implements the semantics of apache/paimon#9037 rather than released Java: `null` and `Long.MIN_VALUE` are treated uniformly as "missing", and there is no early return based on the latest snapshot's value. A latest-value check is unsound for mixed histories — `[100, Long.MIN_VALUE]` is exactly that shape: the latest snapshot carries no watermark, yet an earlier real watermark satisfies the query, so the selector contract requires snapshot 1 to be found. This is the same conclusion as your review on #9037: "[100, null] must still find the earlier valid watermark." The sentinel case follows identically once both are treated as missing. (For `[100, null]` there is in fact no released-Java behavior to mirror at all — the fast path unboxes `snapshot(latest).watermark()` and NPEs.) So the two PRs are a pair: #9037 fixes the Java searches, and this PR implements the same semantics in Rust from the start, so no released Rust version ever disagrees with Java on this lookup. The other documented deviation (returning the walked-back watermark-bearing snapshot instead of the raw midpoint, which may carry no watermark at all) is likewise aligned with #9037. My proposal: hold this PR until #9037 is merged — at that point the Rust behavior matches the Java implementation by construction, including this exact case. If you'd prefer not to gate Rust on the Java fix, I can instead reinstate the latest-sentinel early return here to mirror released Java, and drop it in a follow-up once #9037 lands. Which sequencing would you prefer? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
