Rich-T-kid opened a new pull request, #26579: URL: https://github.com/apache/datafusion/pull/26579
## Which issue does this PR close? - Closes #26530. - Part of #24111. - Depends on apache/arrow-rs#11780. ## Rationale for this change With `binary_as_string` and `enable_rle_to_dictionary` enabled, unannotated Parquet binary strings are expanded and rebuilt as a string dictionary. This showed up as a conversion regression in ClickBench. ## What changes are included in this PR? - Coerce the file schema directly from `Binary` to `Dictionary<K, Utf8>`. - Pin Arrow PR #11780 so the Parquet reader constructs the requested string dictionary directly, without an intermediate dictionary cast. - Extend the opener regression test to verify the resulting keys and values. This is the focused reader/schema portion of the closed draft #26250. ## What is the testing strategy for this PR? - `cargo test -p datafusion-datasource-parquet --lib` - `cargo clippy --all-targets --all-features -- -D warnings` - Workspace format and typo checks - Hosted partitioned ClickBench comparison with both Parquet options enabled (pending) ## Are there any user-facing changes? Parquet scans preserve RLE dictionaries when binary columns are requested as strings, avoiding the expand-and-rebuild conversion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
