mbutrovich opened a new issue, #11836: URL: https://github.com/apache/arrow-rs/issues/11836
### Describe the bug Since #11209, extending a union with nulls returns an error when no field is nullable ([union.rs](https://github.com/apache/arrow-rs/blob/01a9917658fecd639991e92a5a5810be710b539c/arrow-data/src/transform/union.rs#L22-L34)). A struct or fixed-size list extends every child with nulls ([structure.rs](https://github.com/apache/arrow-rs/blob/01a9917658fecd639991e92a5a5810be710b539c/arrow-data/src/transform/structure.rs#L46-L53), [fixed_size_list.rs](https://github.com/apache/arrow-rs/blob/01a9917658fecd639991e92a5a5810be710b539c/arrow-data/src/transform/fixed_size_list.rs#L58-L68)), so it now fails on such a child, although its own validity can hold the null. `merge_n` with a `None` index reaches it. ### To Reproduce ```rust use std::sync::Arc; use arrow_array::{Array, ArrayRef, Int32Array, StructArray, UnionArray}; use arrow_data::transform::MutableArrayData; use arrow_schema::{DataType, Field, UnionFields}; use arrow_select::merge::merge_n; fn main() { // A struct whose child is a union with no nullable field let fields = UnionFields::try_new([0], [Field::new("a", DataType::Int32, false)]).unwrap(); let union = UnionArray::try_new(fields, vec![0].into(), None, vec![Arc::new(Int32Array::from(vec![1])) as ArrayRef]) .unwrap(); let s = StructArray::from(vec![( Arc::new(Field::new("u", union.data_type().clone(), false)), Arc::new(union) as ArrayRef, )]); println!("{:?}", merge_n(&[&s as &dyn Array], &[Some(0), None])); // Err(InvalidArgumentError("struct column 0 (\"u\") failed: Invalid argument error: Cannot extend a union with no nullable fields with nulls")) let data = s.to_data(); let mut m = MutableArrayData::new(vec![&data], true, 2); m.extend(0, 0, 1); println!("{:?}", m.try_extend_nulls(1)); // the same error } ``` ### Expected behavior `[{u: 1}, null]` from `merge_n`, and `Ok(())` from `try_extend_nulls`. ### Additional context This is the `MutableArrayData` form of #11608, which #11731 fixes for `take`. Found with property-based tests that build arrays in every physical layout the Arrow format allows, on `main` at `01a991765`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
