XiaoHongbo-Hope opened a new pull request, #10336: URL: https://github.com/apache/paimon/pull/10336
## Purpose Avoid materializing a complete wide VARIANT column in Python when callers only need a small typed field set. Depends on apache/paimon-rust#1009. ## API Extends the existing ReadBuilder.with_projection API with an optional variant_fields mapping. Each VARIANT column specifies paths, target_type, and fail_on_error. The result column is an Arrow struct with children named 0, 1, ... in requested path order. The option is forwarded through batch, streaming, and Ray read bridges. If native reading is disabled or the installed pypaimon-rust lacks the API, PyPaimon raises a clear error instead of silently falling back to full Python VARIANT decoding. ## Validation - 138 focused tests passed, plus 46 subtests - Flake8 passed for all changed Python files - End-to-end performance A/B against the user workload is running separately and will be added before marking ready -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
