schenksj opened a new issue, #12330:
URL: https://github.com/apache/arrow-rs/issues/12330

   **Describe the bug**
   
   The DELTA_LENGTH_BYTE_ARRAY and DELTA_BYTE_ARRAY decoders read the value 
count from each
   DELTA_BINARY_PACKED length header (lengths, prefix lengths, suffix lengths) 
and size a `Vec<i32>`
   from it before reading any lengths:
   
   - `encodings::decoding::DeltaLengthByteArrayDecoder::set_data` 
(`self.lengths.resize(num_lengths, 0)`)
   - `encodings::decoding::DeltaByteArrayDecoder::set_data` 
(`self.prefix_lengths.resize(num_prefixes, 0)`)
   - `arrow::decoder::DeltaByteArrayDecoder::new` (`vec![0; num_prefix]`, 
`vec![0; num_suffix]`), used by
     the byte array, byte view and fixed length byte array readers
   - `arrow::array_reader::byte_array::ByteArrayDecoderDeltaLength::new` and
     `byte_view_array::ByteViewArrayDecoderDeltaLength::new` (`vec![0; values]`)
   
   The count is not checked against the page's value count or against the 
page's size. A page of a
   few bytes declaring 2^60 values panics with a capacity overflow. A page 
declaring 2^31 values asks
   for 8 GiB, which aborts the process or gets it killed when memory is 
limited, before the decoder
   finds there is no data.
   
   **To Reproduce**
   
   A DELTA_LENGTH_BYTE_ARRAY page whose data is only a DELTA_BINARY_PACKED 
header (block size 128,
   4 mini blocks, total count `1 << 60`, first value 0), read through 
`ByteArrayColumnValueDecoder`
   with `set_data(Encoding::DELTA_LENGTH_BYTE_ARRAY, page, 4, Some(4))`.
   
   **Expected behavior**
   
   A `ParquetError` naming the declared and allowed counts, before anything is 
sized from the count.
   
   **Additional context**
   
   Present in 59.3.1, 60.1.0 and `main`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to