JingsongLi opened a new pull request, #831: URL: https://github.com/apache/paimon-rust/pull/831
### Purpose Related: #413 (follow-up to the Python scan-planning bindings). Make native scan plans usable by PyPaimon for incremental and distributed reads while preserving snapshot identity, branch isolation, deletion vectors, and primary-key merge groups. For example, versions of the same primary key in two APPEND snapshots must be planned together; planning each commit separately can return both versions. ### Brief change log - Add combined APPEND-delta planning for an explicit snapshot range, using the ending snapshot for metadata and deletion vectors. - Add Data Evolution row-position slices and shards before group-statistics pruning, projection, and deletion-vector filtering. Preserve group order when a later limit selects rows. - Preserve snapshot IDs for empty plans, expose raw scan traces and runtime capability markers, and resolve branch-qualified table handles and their metadata managers. - Decode deletion-vector arrays using their actual Avro writer schema and read legacy Python bucket-local index references from `table/index` when the canonical path is absent. - Accept Python `bytes`/`bytearray` binary literals and keep overlapping composite floating-point key ranges together, including signed zero and NaN bounds. ### Tests Passed locally: - Rust table-scan tests: 100. - Rust incremental batch-scan tests: 31. - Rust row-position selection and index-path tests: 5 and 13. - Python binding read/table tests against a newly built wheel: 89. - `cargo check -p pypaimon_rust`, `cargo fmt --all -- --check`, and `git diff --check`. The wheel was also validated with companion PyPaimon changes in a separate checkout: 193 planner-parity regressions and 196 reader regressions passed, including ORC. Those companion Python changes are outside this PR. The full workspace and Spark-warehouse integration suites were not run locally. ### API and Format Adds `Plan.snapshot_id()`, `Table.branch()`, `TableScan.plan_with_trace()`, `ReadBuilder.new_incremental_scan(start, end)`, DE row-position selection methods, and `planning_capabilities()` to the binding, plus the corresponding combined-scan APIs in the core. Existing per-commit `IncrementalScan.plan()` behavior is preserved. The newly added row-position configuration requires combined planning. Split/storage format versions are unchanged. NaN key bounds use conservative overlapping groups. Legacy bucket-local DV resolution can add filesystem existence checks; explicit external paths and existing canonical files retain priority. Complete parity for all PyPaimon modes remains follow-up work. ### Documentation Updated `docs/src/python-binding.md` and the Python type stubs with the new APIs, range semantics, and compatibility guarantees. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
