b84330808 opened a new pull request, #55389: URL: https://github.com/apache/arrow/pull/55389
### Rationale for this change Out-of-bounds dictionary indices are rejected by `DictDecoder::Decode()` and `DecodeArrow()`, but: - `DecodeIndices()` and `DecodeIndicesSpaced()` don't check them at all, so `ReadBatchWithDictionary()` and `read_dictionary` reads can return indices past the end of the dictionary; - `RleBitPackedDecoder` doesn't stop when a run it is resuming (or a long bit-packed run) has an invalid index: it moves on to the next run. That silently skips values, and for an RLE run it loops forever in release builds (and hits a DCHECK in debug builds). ### What changes are included in this PR? - `RleBitPackedDecoder::ProcessValues()` and `GetSpaced()` stop as soon as a run is not entirely consumed while the batch is not full, which only happens on invalid input, instead of resuming with the next run. - `DictDecoderImpl::DecodeIndices()` (both overloads) and `DecodeIndicesSpaced()` check the decoded indices against the dictionary length, like the other dictionary decoding methods do. ### Are these changes tested? Yes, new tests in `encoding_test.cc` (all `DictDecoder` methods with an out-of-bounds index) and `rle_encoding_test.cc` (resuming a partially read RLE run). Without the fix, the Parquet tests fail and the RLE test hits the DCHECK in debug builds; in release builds it doesn't return. ### Are there any user-facing changes? Reading Parquet data with out-of-bounds dictionary indices now errors out in these code paths. **This PR contains a "Critical Fix".** Invalid dictionary indices could make the decoder silently return values from the following run (incorrect data) or loop forever. ### Was AI used for this PR? In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR. **PR code and description written by:** - [ ] Human - [x] AI **Reviewed before submission by:** - [x] Human - [x] AI - [ ] Not reviewed I used Claude Code to find this (with a local fuzz harness over the RecordReader / low-level reader APIs) and to write the change and tests. * GitHub Issue: #55388 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
