DarkWanderer opened a new pull request, #12321: URL: https://github.com/apache/arrow-rs/pull/12321
# Which issue does this PR close? - None filed. Found while benchmarking dictionary-based row group pruning in apache/datafusion#23851. # Rationale for this change `ParquetRecordBatchStreamBuilder::get_column_chunk_dictionary`, `ParquetMetaDataReader::read_column_dictionary` and `ParquetMetaDataReader::read_column_dictionary_async` fail with `Dictionary page encoding must be PLAIN, got PLAIN_DICTIONARY` when a dictionary page header declares the deprecated `PLAIN_DICTIONARY` encoding. Older writers label dictionary pages that way, for example parquet-cpp 1.x, which wrote the ClickBench `hits` files. The array readers already decode such pages as `PLAIN`, so the column data of these files reads fine, but their dictionaries cannot be read through these APIs. # What changes are included in this PR? `decode_dictionary_page` accepts `PLAIN_DICTIONARY` as well as `PLAIN` for the dictionary page encoding, as the array readers do. Other encodings are still rejected. # Are these changes tested? Yes. A unit test rewrites a dictionary page header to `PLAIN_DICTIONARY` and checks that the dictionary decodes to the expected values. The existing test that rejects `RLE_DICTIONARY` is unchanged. # Are there any significant user-facing changes? No. # AI usage disclosure The fix, the test and this description were written with Claude Code (Claude Opus 5.5). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_012oZ19LDPpsfabaCZBZHoDu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
