hhhizzz opened a new pull request, #11517: URL: https://github.com/apache/arrow-rs/pull/11517
# Which issue does this PR close? - Closes #11516. # Rationale for this change A corrupt RLE / bit-packed hybrid run header can decode to any 64-bit value. `RleDecoder::reload` multiplied the bit-packed group count by 8 in `i64` without a check, so reading such a page panics with `attempt to multiply with overflow` when overflow checks are enabled (e.g. debug builds). Otherwise the length wraps, decoding continues with a bogus run length, and the read fails later with an unrelated error or panics elsewhere. `PackedDecoder`, the definition level fast path, parses run headers itself and has the same unchecked multiplication. # What changes are included in this PR? - Add `run_len`, which returns the number of values in a run, or an error if it is above 2^31 - 1, the maximum run length allowed by the [Parquet spec](https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding--bit-packing-hybrid-rle--3). Arrow C++ enforces the same limit. - Use it in `RleDecoder::reload` and `PackedDecoder::next_rle_block`. # Are these changes tested? Yes. `test_run_len_limit` checks that the longest runs allowed by the spec are accepted and that longer ones are rejected, including headers whose length overflows `i64` and headers that are negative as `i64`. `test_packed_decoder_invalid_run_len` covers `PackedDecoder`. Both tests fail without this change. # Are there any significant user-facing changes? No. --- This PR was written with the help of Claude Code and Codex: the fix, the tests and this description were AI-generated. I have reviewed the change, and it was verified locally, including against the corrupt file that first triggered the panic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
