Rich-T-kid opened a new issue, #10951:
URL: https://github.com/apache/arrow-rs/issues/10951
### Describe the bug
`ArrayData` struct validation permits nulls in non-nullable child fields
when those child nulls are masked by a null parent struct row. When the parent
`Struct` `ArrayData` has a non-zero offset, parent row `i` maps to child index
`offset + i`.
The current validation path appears to compare the parent null bitmap
directly against the child null bitmap without applying the struct parent
offset:
```rust
self.validate_non_nullable(self.nulls(), child)?
```
That makes validation use different coordinate systems for parent and child
nulls. It rejects a child null that is actually masked by the parent, and
accepts a child null that is visible through a non-null parent row.
Related to #10933 and #10934.
### To reproduce
Add this test to `arrow-data/src/data.rs` inside `mod tests`:
```rust
#[test]
fn test_struct_non_nullable_child_nulls_account_for_parent_offset() {
let build = |parent_nulls| {
let child = ArrayData::builder(DataType::Int32)
.len(5)
.add_buffer(Buffer::from_slice_ref([0, 1, 2, 3, 4]))
.nulls(Some(NullBuffer::new(BooleanBuffer::from(vec![
true, true, false, true, true,
]))))
.build()
.unwrap();
ArrayData::builder(DataType::Struct(Fields::from(vec![Field::new(
"x",
DataType::Int32,
false,
)])))
.len(4)
.offset(1)
.nulls(Some(NullBuffer::new(BooleanBuffer::from(parent_nulls))))
.add_child_data(child)
.build()
};
// Parent row 1 maps to child index 2, so this parent null masks the
child null.
assert!(build(vec![true, false, true, true]).is_ok());
// Parent row 2 maps to child index 3, leaving the child null at index 2
unmasked.
assert!(build(vec![true, true, false, true]).is_err());
}
```
Run:
```shell
cargo test -p arrow-data --lib
test_struct_non_nullable_child_nulls_account_for_parent_offset
```
### Expected behavior
With `Struct(offset=1, len=4)`, the parent null bitmap describes the visible
parent rows, while child nulls are in child coordinates:
- parent row 1 maps to child index 2, so a parent null at row 1 should mask
a child null at index 2 and validation should pass
- parent row 2 maps to child index 3, so it should not mask a child null at
index 2 and validation should fail
### Actual behavior
Verified on current `main` at:
```text
6e728ced77368fa8a947ac52e7efba19fb5dfde9 Fix pool claim race condition
(#10301)
```
The observed behavior is reversed:
```text
parent null masks child index 2: Err(InvalidArgumentError("non-nullable
child of type Int32 contains nulls not present in parent"))
parent null masks child index 3, not child index 2: Ok(...)
```
So the first assertion fails today, and the second assertion would currently
pass for the wrong reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
