Rich-T-kid opened a new pull request, #12161: URL: https://github.com/apache/arrow-rs/pull/12161
# Which issue does this PR close? - Closes #12160. # Rationale for this change A Parquet `BYTE_ARRAY` column without a UTF8 annotation can be requested as `Dictionary(Int32, Utf8)`. The dictionary reader currently decodes its dictionary values as `Binary`, so callers must cast whole batches after decoding. Pushed predicates can consequently rebuild the dictionary for every batch before evaluating a string predicate. # What changes are included in this PR? The byte-array dictionary reader decodes into the physical string or binary value type, then converts each dictionary page values once to the requested compatible type while preserving the encoded keys. The converted values are reused across batches backed by the same dictionary page. Plain-encoded fallback pages retain the existing dictionary-building path. # Are these changes tested? Coverage includes binary dictionary pages requested as `Utf8`, `LargeUtf8`, and `Utf8View`, reuse across batches, nulls, invalid UTF-8 rejection, and plain-encoded fallback pages. A draft DataFusion dependency PR will provide the end-to-end pushed-predicate benchmark evidence before this PR is marked ready. # Are there any significant user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
