JingsongLi opened a new pull request, #10392: URL: https://github.com/apache/paimon/pull/10392
## Purpose Follow up the final review of #10336 by making the structured `read_type` the single reader request for PyPaimon and Native. Depends on https://github.com/apache/paimon-rust/pull/1041. ## Changes - Remove ReadBuilder's `_projection`, `_nested_paths` and `_resolved_read_type` state. `with_projection` compiles the canonical nested read type, while a separate result projection contains aliases, duplicate outputs and flattening order. - Preserve source ROW field IDs and structure. Selected MAP values use the same temporary ROW description metadata as Java; nested VARIANT extraction paths, target types and strict/try behavior are included in the read type. - Use only `with_read_type` for Native reads and Native planning. Remove the separate `nested_projection` transport. - Pass the same canonical type through batch, stream and Ray split providers. Derive the local Python format adapter from that type, and apply result projection separately. Update Temporal and Torch schema consumers to use result fields. - Support named nested ROW/MAP selectors and nested VARIANT expressions. Preserve parent nulls, whole-column precedence, output aliases/order/duplicates and empty projections. - Add real Native end-to-end tests across append, primary-key and data-evolution modes; shared MAP shredding, plain/shredded VARIANT, schema/provider reconciliation, partial-update MAP aggregates, reverse nested field order, and structured `with_read_type` results. Java is the reference for reader field IDs, selected-key MAP metadata and VARIANT physical clipping. Native implementation is in Rust Core, with the binding accepting the complete schema JSON. ## Validation Built the binding from apache/paimon-rust#1041, then ran the affected read/plan/update, projection, MAP/VARIANT, stream, Ray, Temporal, schema-evolution and query-authorization suites with all five Native CI paths enabled: - **1230 passed, 6 skipped, 218 subtests passed**. - Native counters: **1489 plans, 1691 reads, 904 writes, 36 commits**; updates: 97 row-id, 53 grouped, 63 predicate, 27 upsert, 27 incremental. - Changed Python sources: flake8 passed and Python 3.6 grammar check passed. - Companion Rust Core: 3699 tests passed; full workspace Clippy with `-D warnings`, rustfmt and 113 binding tests passed. Torch is not installed locally; its changed schema consumer was inspected but the Torch suite was not run. This PR is draft until the Rust dependency lands. CI still follows `apache/paimon-rust@main`; no fork or temporary revision pin is added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
