shashank11yadav opened a new pull request, #12324: URL: https://github.com/apache/arrow-rs/pull/12324
# Which issue does this PR close? - Closes #12290 # Rationale for this change A Parquet page encoded with DELTA_LENGTH_BYTE_ARRAY (including the suffix stream of DELTA_BYTE_ARRAY) whose lengths are negative, or add up to more bytes than the page holds, makes the reader panic instead of returning an error. A negative length gives an out-of-range offset in `get`, and a length past the end panics in `Bytes::slice`. Malformed input should return a ParquetError. # What changes are included in this PR? - `DeltaLengthByteArrayDecoder::set_data` in `parquet/src/encodings/decoding.rs` now checks the decoded lengths when the page is set. A negative length, a sum that overflows, or a sum greater than the data after the length stream returns an error. - `DeltaByteArrayDecoder` uses the same decoder for its suffixes, so it gets the same check. - Three tests in the module's `mod tests` cover a negative length, a length past the end of the data, and a negative suffix length. `get` and `skip` are unchanged. The checks in `set_data` make their offsets valid. # Are these changes tested? The three new tests fail without this change and pass with it. The decoding module tests, clippy with warnings denied, the rustfmt check, and the arrow-feature type check all passed on this commit. Not run: the typos spell check, and the full parquet test suite. Only the decoding module tests were run. # Are there any significant user-facing changes? No public API change. Malformed pages that used to panic now return a ParquetError. # AI usage disclosure I used an AI coding assistant while preparing this change; I reviewed and tested all of it myself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
