etseidl commented on code in PR #10979:
URL: https://github.com/apache/arrow-rs/pull/10979#discussion_r3935006586
##########
parquet/src/parquet_thrift.rs:
##########
@@ -721,7 +733,20 @@ where
{
let list_ident = prot.read_list_begin()?;
validate_list_type(T::ELEMENT_TYPE, &list_ident)?;
- let mut res = Vec::with_capacity(list_ident.size as usize);
+ let size = list_ident.size as usize;
+ // Each list element occupies at least one byte on the wire. Bound the
+ // declared count by remaining input before reserving, so a malformed
+ // header cannot abort the process with a huge allocation.
+ if let Some(remaining) = prot.remaining_bytes()
Review Comment:
Thanks, this is a sensible bound. Previous attempts to address this would
factor in the data size resulting in errors on compressed data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
