ump45nose opened a new pull request, #11697: URL: https://github.com/apache/arrow-rs/pull/11697
# Which issue does this PR close? - Closes #11690. # Rationale for this change `ParquetMetaDataPushDecoder::try_decode` computed the metadata start as `file_len - footer_len - metadata_len` without checking the footer's metadata length against the file length. A corrupt footer that claims more metadata than the file holds made that subtraction underflow: it panicked with overflow checks on, and without them the wrapped value later panicked in `Bytes::slice`. `ParquetMetaDataReader` already returns an EOF error for the same bytes. # What changes are included in this PR? `try_decode` in `parquet/src/file/metadata/push_decoder.rs` uses `checked_sub` for the metadata start and returns the same `EOF: Parquet file too small. Size is N but need M` error as `ParquetMetaDataReader` when the footer and metadata don't fit in the file. # Are these changes tested? Yes. `test_metadata_decoder_metadata_length_exceeds_file` builds the 12-byte file from the issue (4 data bytes and a footer claiming 1000 bytes of metadata) and checks that `try_decode` returns `EOF: Parquet file too small. Size is 12 but need 1008`. Without the fix the test panics with `attempt to subtract with overflow`. I also ran `cargo test -p parquet --lib push_decoder`, `cargo fmt -p parquet -- --check` and `cargo clippy -p parquet --lib --tests -- -D warnings`. # Are there any significant user-facing changes? No. Corrupt files that used to panic now return an error. AI disclosure: I used an AI coding assistant to write the fix, the test and this description, and I reviewed all of it. The part worth a reviewer's look is that the decoder is left in its intermediate state after this error, the same as for the other decode errors in `try_decode` (for example a bad footer magic), rather than being restored so the call can be retried. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
