JingsongLi opened a new pull request, #1041: URL: https://github.com/apache/paimon-rust/pull/1041
## Summary Make the structured Paimon read type authoritative for nested ROW, selected MAP keys and VARIANT extraction. This is the Rust half of the requested follow-up to apache/paimon#10336. - Move nested request construction and Java selected-key MAP metadata into Rust Core. Temporary MAP ROW children retain the complete MAP value type; keys are matched by metadata ordinal. - Drive Parquet physical clipping from read fields, including nested ROW leaves, shared MAP direct/overflow columns, and case-sensitive shredded VARIANT object/array paths. - Resolve clipped shredded VARIANT paths directly, retaining fallback value columns when required, and preserve decoded field names/order before field-ID reconciliation. - Apply synthetic MAP/VARIANT requests across append, primary-key and data-evolution readers. Preserve original source values for effective partial-update aggregates before projecting selected keys. - Validate selected MAP read types against the logical schema and query-authorization grants. Preserve synthetic field IDs through nested data-evolution provider selection. - Remove the Python binding's independent `with_nested_projection` API. `with_read_type` carries the complete request; callers handle result aliases and flattening. - Cover MAP<BLOB> providers and Java row-tracking sequence fallback while using selected-key read types. ## Java references `ReadBuilderImpl`, `MapSelectedKeysMetadataUtils`, `MapSharedShreddingReadPlanFactory`, `VariantShreddingTypePruner` and `DataFileRecordReader` in current Apache Paimon master. ## Validation - `cargo test --locked -p paimon --lib`: **3699 passed, 6 ignored**. - `cargo clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings`: passed. - `cargo fmt --all -- --check`: passed. - Python binding regressions: **113 passed**. - Built this branch's binding and ran PyPaimon with all five Native CI paths enabled: **1230 passed, 6 skipped, 218 subtests passed** across nested projections, MAP/VARIANT, stream/Ray/Temporal, schema evolution, authorization and updates. Native counters: 1489 plans, 1691 reads, 904 writes, 36 commits, plus row-id/grouped/predicate/upsert/incremental updates. The companion PyPaimon PR continues using `apache/paimon-rust@main` in CI and depends on this change being merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
