Tembocs opened a new issue, #10921:
URL: https://github.com/apache/arrow-rs/issues/10921
### Describe the bug
`arrow_ipc::reader::read_block` (reader.rs:922) sizes its buffer from the
footer's `Block` before reading:
let body_len = block.bodyLength().to_usize().unwrap();
let metadata_len = block.metaDataLength().to_usize().unwrap();
let total_len = body_len.checked_add(metadata_len).unwrap();
let mut buf = MutableBuffer::from_len_zeroed(total_len);
reader.read_exact(&mut buf)?;
A 1,018-byte file whose footer holds a record-batch block with
`bodyLength = 0x280000000240` (44 TB) reaches `from_len_zeroed`, which
ASan reports as allocation-size-too-big and which aborts the process on
a normal build. `read_exact` would have returned `UnexpectedEof` had the
allocation been bounded — the check exists one line too late.
`FileReader::try_new` verifies the footer's flatbuffer but not that its
blocks lie within the file.
### To Reproduce
```
cargo new ipc-repro && cd ipc-repro
cargo add [email protected]
# copy the attached block-body.arrow beside Cargo.toml
// src/main.rs
fn main() {
let file = std::fs::File::open("block-body.arrow").unwrap();
let reader = arrow_ipc::reader::FileReader::try_new(file,
None).unwrap();
for batch in reader {
println!("{:?}", batch.map(|b| b.num_rows()));
}
}
```
cargo run --release
### Expected behavior
`Err(ArrowError::IpcError(..))` — the block is beyond the file.
### Additional context
`FileReader` knows the file's length (it seeks to the footer). Checking
each block's `offset + metaDataLength + bodyLength <= file_len` when the
footer is parsed — or reading with a length-capped `take` into a growing
buffer in `read_block` — turns this into an error. The `MessageReader`
path (reader.rs:1845, `from_len_zeroed(message.bodyLength())`) has the
same shape for the stream format.
Linux x86_64, Rust stable 1.98.0. Found by fuzzing a reader built on the
crate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
