adriangb opened a new issue, #12249:
URL: https://github.com/apache/arrow-rs/issues/12249

   ## Describe the bug
   
   The IPC writer writes a `RunEndEncoded` array with `Utf8View` (or 
`BinaryView`) values that is sliced to zero rows, but the reader cannot read it 
back. The values must have at least one non-inlined string (longer than 12 
bytes), so that they own a data buffer. The same happens when the view array is 
nested inside the run values, for example `RunEndEncoded<Int32, 
ListView<Utf8View>>` or `RunEndEncoded<Int32, List<Utf8View>>`.
   
   ## To Reproduce
   
   Put this file in `arrow-ipc/tests/ree_repro.rs` and run `cargo test -p 
arrow-ipc --test ree_repro -- --nocapture`.
   
   ```rust
   use arrow_array::types::Int32Type;
   use arrow_array::*;
   use arrow_ipc::{reader::StreamReader, writer::StreamWriter};
   use std::sync::Arc;
   
   #[test]
   fn repro() {
       let values = StringViewArray::from(vec!["a long string that is not 
inlined", "b"]);
       let ree = RunArray::<Int32Type>::try_new(&Int32Array::from(vec![1, 2]), 
&values).unwrap();
       let batch = RecordBatch::try_from_iter([("a", Arc::new(ree) as 
ArrayRef)])
           .unwrap()
           .slice(1, 0);
   
       let mut buf = Vec::new();
       let mut writer = StreamWriter::try_new(&mut buf, 
&batch.schema()).unwrap();
       writer.write(&batch).unwrap();
       writer.finish().unwrap();
       drop(writer);
   
       let read = StreamReader::try_new(buf.as_slice(), 
None).unwrap().next().unwrap();
       println!("{:?}", read.map(|b| b.num_rows()));
   }
   ```
   
   ## Expected behavior
   
   The reader returns a batch with 0 rows: `Ok(0)`.
   
   Actual output:
   
   ```
   Err(IpcError("Invalid variadic count 1 for Utf8View column, with 2 buffer(s) 
left in the message"))
   ```
   
   ## Additional context
   
   Version: `main` at 
https://github.com/apache/arrow-rs/commit/df096d7cf042b645a7e35759ca865bedbd98ea92
   
   Suspected cause: the buffers and the variadic buffer counts come from two 
different arrays.
   
   - For a run array, the writer writes the children of 
`unslice_run_array(array_data)`: 
https://github.com/apache/arrow-rs/blob/df096d7cf042b645a7e35759ca865bedbd98ea92/arrow-ipc/src/writer.rs#L2667-L2681
   - For an empty run array, `into_zero_offset_run_array` returns 
`ArrayData::new_empty`, whose `Utf8View` values have no data buffers: 
https://github.com/apache/arrow-rs/blob/df096d7cf042b645a7e35759ca865bedbd98ea92/arrow-ipc/src/writer.rs#L1321-L1324
   - `append_variadic_buffer_counts` walks the original `array_data`, so it 
counts the data buffer of the original values: 
https://github.com/apache/arrow-rs/blob/df096d7cf042b645a7e35759ca865bedbd98ea92/arrow-ipc/src/writer.rs#L1200
 and 
https://github.com/apache/arrow-rs/blob/df096d7cf042b645a7e35759ca865bedbd98ea92/arrow-ipc/src/writer.rs#L1277-L1293
   
   A possible fix is to compute the variadic counts from the same unsliced run 
array that the writer writes.
   
   AI-assisted, verified by running the reproducer.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to