JingsongLi opened a new pull request, #9873: URL: https://github.com/apache/paimon/pull/9873
### Purpose Expand native planning across resolved table contexts and scan modes while preserving Java semantics. This draft depends on https://github.com/apache/paimon-rust/pull/847. Native CI continues to build Rust main; the dependency must merge before the new native coverage can pass there. - Build filesystem-backed native tables from the Python table's resolved schema and complete effective options. This enables stale schemas, `copy()` option overrides/removals, and local `FileStoreTable.from_path()` reads without reloading the catalog. REST tables retain their catalog credential and snapshot behavior; unreproducible contexts retain fallback. - Use Rust file and deletion-vector planning for append/DE chunk shuffle, including incremental windows. Keep Python's live-row chunk sizing, seeded ordering and balanced chunk assignment, preserve resolved file paths, and retain aligned column files before chunk construction. - Enable native planning for fully materialized first-row/DV clustering files across levels. First-row plans containing L0 still fall back pending the mixed clustered-file reader work. - Fix Python raw readers losing filters after schema evolution: physical-name pushdown cannot safely identify renamed/re-added columns, so cross-schema predicates are evaluated after field-ID normalization and before projection/limit. Correct the older test that allowed NULL-valued old rows through an AND equality filter. - Fix `FileStoreTable.from_path()` to use the FileIO factory and document the expanded native coverage. Precomputed PK global-index results, query authorization, continuous streaming and write planning keep their existing Python paths. ### Tests - Rebuilt the wheel from the Rust dependency and rebased on current master. - 352 native/planner/schema-evolution/VARIANT tests passed, exercising 506 native plans. Newly enabled paths explicitly fail if Python planning is used as fallback. - 168 tests passed in the normal Python lane, including schema evolution, predicates, chunk shuffle and native adapter unit tests. - New coverage includes append/PK/DE stale schemas, rename and drop/re-add field identity, removed selectors/search options, dynamic merge-engine options, catalogless paths/file URIs, custom contexts, cross-level clustered/DV raw splits, chunk live-row counts/order/worker assignment, partition filters, blob siblings, time travel and incremental events. - The clustered materialized-DV regression fails with the previous Rust wheel. The new schema-evolution filter cases reproduced the Python reader error before the fix. - Flake8 for all changed Python files and `git diff --check` passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
