Tembocs opened a new issue, #10920:
URL: https://github.com/apache/arrow-rs/issues/10920
### Describe the bug
`parquet_thrift::read_thrift_vec` (parquet_thrift.rs:724) reserves the
result vector from the list header's declared size before reading any
element:
let list_ident = prot.read_list_begin()?;
validate_list_type(T::ELEMENT_TYPE, &list_ident)?;
let mut res = Vec::with_capacity(list_ident.size as usize);
Decoding `FileMetaData` from a 23-byte file whose `schema` list header
declares 109,002,364 elements asks for `109,002,364 ×
size_of::<SchemaElement>()`
= 10,464,226,944 bytes. An allocation failure is an abort, not an error:
`ParquetMetaDataReader` never returns, and the caller's process dies. The
list size is already validated to fit an `i32` (`read_list_begin`), but
not against the bytes available, which is the bound that matters.
### To Reproduce
```
cargo new parquet-repro && cd parquet-repro
cargo add [email protected] --no-default-features
# copy the attached schema-list.parquet beside Cargo.toml
// src/main.rs
use parquet::file::metadata::ParquetMetaDataReader;
fn main() {
let file = std::fs::File::open("schema-list.parquet").unwrap();
let r = ParquetMetaDataReader::new().parse_and_finish(&file);
println!("{:?}", r.map(|_| ()));
}
```
cargo run --release
### Expected behavior
`Err(ParquetError::General(..))` — the footer is malformed.
### Additional context
Every list element occupies at least one byte on the wire, so
`list_ident.size` can be bounded by the bytes remaining in the metadata
slice before reserving (or the capacity capped and the vector grown as
elements decode). The same pattern applies to `read_bytes` for binary
fields and to map sizes. The footer's declared length is already bounded
by the file in `ParquetMetaDataReader`; this is the one level below.
Linux x86_64, Rust stable 1.98.0. Found by fuzzing a reader built on the
crate.
[schema-list.parquet.zip](https://github.com/user-attachments/files/31626459/schema-list.parquet.zip)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
