kawadakk opened a new issue, #4602: URL: https://github.com/apache/arrow-rs/issues/4602
**Describe the bug** Even if a field is non-nullable in its schema definition, it's only semantic and should not [prevent](https://github.com/apache/arrow-rs/blob/414235e7630d05cccf0b9f5032ebfc0858b8ae5b/arrow-array/src/array/list_array.rs#L126-L132) the creation of arrays containing null values. Null values in semantically non-nullable fields may arise from, for example, being nested inside another nullable field (as in <https://github.com/apache/arrow-rs/issues/3900>) or a sparse union field. [Arrow Columnar Format](https://arrow.apache.org/docs/format/Columnar.html#schema-message): > The `Field` Flatbuffers type contains the metadata for a single array. This includes: \[...\] Whether the field is semantically nullable. While this has no bearing on the array’s physical layout, many systems distinguish nullable and non-nullable fields and we want to allow them to preserve this metadata to enable faithful schema round trips. **To Reproduce** ```rust let field = Arc::new(Field::new("element", DataType::Int32, false)); let offsets = OffsetBuffer::new(vec![0, 1, 4, 5].into()); let values = new_null_array(&DataType::Int32, 5); ListArray::new(field, offsets, values, None); ``` The above code panics with message ``called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Non-nullable field of ListArray \"element\" cannot contain nulls")`` **Expected behavior** Not panicking **Additional context** <!-- Add any other context about the problem here. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
