JingsongLi commented on PR #979: URL: https://github.com/apache/paimon-rust/pull/979#issuecomment-5934762451
[P2] Include `scan.watermark` in the time-travel guard used by DROP PARTITION The new check at `crates/integrations/datafusion/src/sql_context.rs:2031` still allows a supported historical selector: `ensure_no_time_travel_for_write` checks five keys but omits `SCAN_WATERMARK_OPTION`. This head already applies `SET 'paimon.scan.watermark'` through `copy_with_time_travel` to SELECT providers, while DROP retrieves the raw latest table and can commit normally. I reproduced this with a native partitioned table containing two committed snapshots, with valid snapshot watermarks 100 and 200 (the metadata Java writers produce). After `SET 'paimon.scan.watermark' = '100'`, SELECT returned the one row in snapshot 1. `ALTER TABLE ... DROP PARTITION (pt = 'b')` nevertheless succeeded; after RESET, the latest table contained one row instead of two. The mutation uses latest state, while the session's read view is historical. The helper omission predates this PR, but it leaves the stated DROP time-travel protection incomplete. Please add `SCAN_WATERMARK_OPTION` to the shared guard and cover rejection without a new snapshot/deletion. A temporary addition of that selector makes the same probe reject DROP and retain both rows. Validation on head `2d76c46691111c85558b0e5ae25254f192975d7f`: 177 SQLContext tests and 23 REST-managed partition SQL tests passed, including a temporary probe verifying the five currently listed selectors block catalog partition deletion. The additional native watermark scenario fails on the unmodified head. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
