shashank11yadav opened a new issue, #12290:
URL: https://github.com/apache/arrow-rs/issues/12290

   ### Describe the bug
   
   Reading a Parquet file whose DELTA_LENGTH_BYTE_ARRAY column contains a 
negative length panics with `attempt to add with overflow` instead of returning 
a `ParquetError`. The length is cast to `usize` without a check, so it becomes 
a huge value.
   
   ### To Reproduce
   
   The program below writes a one-column file with two values and patches one 
byte of the length stream so the second length decodes as -1. It then reads the 
file with `SerializedFileReader::get_row_iter`. It uses only the public API and 
needs no features.
   
   ```rust
   use parquet::basic::{Compression, Encoding};
   use parquet::data_type::{ByteArray, ByteArrayType};
   use parquet::file::properties::{EnabledStatistics, WriterProperties};
   use parquet::file::reader::{FileReader, SerializedFileReader};
   use parquet::file::writer::SerializedFileWriter;
   use parquet::schema::parser::parse_message_type;
   use parquet::schema::types::ColumnPath;
   use std::{fs, panic, sync::Arc};
   
   fn main() {
       let schema = Arc::new(parse_message_type("message m { required binary c; 
}").unwrap());
       let props = WriterProperties::builder()
           .set_dictionary_enabled(false)
           .set_compression(Compression::UNCOMPRESSED)
           .set_statistics_enabled(EnabledStatistics::None)
           .set_column_encoding(ColumnPath::from("c"), 
Encoding::DELTA_LENGTH_BYTE_ARRAY)
           .build();
       let mut buf = Vec::new();
       let mut w = SerializedFileWriter::new(&mut buf, schema, 
Arc::new(props)).unwrap();
       let mut rg = w.next_row_group().unwrap();
       let mut cw = rg.next_column().unwrap().unwrap();
       cw.typed::<ByteArrayType>()
           .write_batch(&[ByteArray::from("a"), ByteArray::from("b")], None, 
None)
           .unwrap();
       cw.close().unwrap();
       rg.close().unwrap();
       w.close().unwrap();
   
       // Length stream for [1, 1]: 80 01 (block) | 04 (miniblocks) | 02 
(count) | 02 (first=1)
       // | 00 (min_delta=0) | 4 zero bit widths.
       // Patch min_delta to zigzag(-2)=3 so the lengths decode as [1, -1].
       let pat = [0x80, 0x01, 0x04, 0x02, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 
b'a', b'b'];
       let pos = buf.windows(pat.len()).position(|w| w == 
pat).unwrap_or_else(|| {
           let p = buf.windows(2).position(|w| w == b"ab").unwrap_or(40);
           panic!("page not found: {:02x?}", &buf[p.saturating_sub(16)..p + 2]);
       });
       buf[pos + 5] = 0x03;
       let path = std::env::temp_dir().join("l1_repro.parquet");
       fs::write(&path, &buf).unwrap();
   
       println!("EXPECTED: Err(..) for a negative length, no panic");
       let res = panic::catch_unwind(|| {
           let reader = 
SerializedFileReader::new(fs::File::open(&path).unwrap())?;
           let mut n = 0;
           for row in reader.get_row_iter(None)? {
               row?;
               n += 1;
           }
           Ok::<_, parquet::errors::ParquetError>(n)
       });
       match res {
           Err(_) => {
               println!("ACTUAL: panicked on malformed page");
               std::process::exit(1);
           }
           Ok(Ok(n)) => {
               println!("ACTUAL: Ok, read {n} rows");
               std::process::exit(1);
           }
           Ok(Err(e)) => println!("ACTUAL: Err({e})"),
       }
   }
   ```
   
   Output on current `main` (debug build):
   
   ```
   EXPECTED: Err(..) for a negative length, no panic
   
   thread 'main' (2) panicked at parquet/src/encodings/decoding.rs:1001:65:
   attempt to add with overflow
   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
   ACTUAL: panicked on malformed page
   ```
   
   The published 60.1.0 release gives the same panic in a debug build, at 
`parquet/src/encodings/decoding.rs:1014:65`.
   
   ### Expected behavior
   
   Reading a corrupt file should return a `ParquetError` for the bad page, not 
panic. The repo's guidelines on `panic` vs `Result` say to return an error for 
invalid input.
   
   ### Additional context
   
   - parquet 60.1.0 and `main` @ 4f1ed42f, rustc 1.99.1, Linux.
   - `DeltaLengthByteArrayDecoder::set_data` in 
`parquet/src/encodings/decoding.rs` never checks the decoded lengths for 
negative values.
   - In `get`, `self.lengths[..] as usize` turns a negative `i32` into a huge 
`usize`, and `self.offset + len` then overflows.
   - From reading the code (not run): `skip` has the same pattern. It sums the 
`i32` lengths and casts the sum with `as usize`.
   
   A possible fix is to return a `general_err!` from `set_data` if any decoded 
length is negative. `get` and `skip` could also use checked arithmetic and 
return an error when offset plus length passes the end of the data. A 
regression test in `decoding.rs`'s `mod tests` could call `set_data` and `get` 
on a page with a negative length and assert an error.
   
   I'm happy to send a PR for this if it sounds right.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to