aaron-seq opened a new pull request, #50546:
URL: https://github.com/apache/arrow/pull/50546
### Rationale for this change
When casting a nested struct whose inner field changes from nullable to
non-nullable, `CastStruct::Exec` currently calls
`in_values->GetNullCount() > 0` without considering the parent struct's
validity bitmap. This causes a spurious `ArrowInvalid` error when the
child's physical nulls are fully masked by null parent entries.
Reported in #50515.
### What changes are included in this PR?
In `scalar_cast_nested.cc`, replace the unconditional rejection with a
three-way check:
1. **Parent has no bitmap** → child nulls are unmasked; reject.
2. **Child has no bitmap** → element-wise check against parent validity.
3. **Both bitmaps present** → use `BinaryBitBlockCounter::NextAndNotWord()`
to detect parent-valid-AND-child-null positions without heap allocation.
### Are these changes tested?
Five new C++ test cases in `scalar_cast_test.cc`:
- `StructNestedNullabilityAbsentParent` — true violation, must reject
- `StructNestedNullabilityMasked` — masked null, must succeed
- `StructNestedNullabilitySliced` — offset correctness for both outcomes
- `StructNestedNullabilityAbsentChild` — no nulls, must succeed
- `StructNestedNullabilityDeep` — 3-level nesting with masked null
One new Python test in `test_compute.py`:
- `test_cast_struct_nested_nullability` — end-to-end PyArrow validation
covering success, rejection, and sliced array cases
### Are there any user-facing changes?
Struct casts that were previously rejected with
`ArrowInvalid: field '...' has nulls` will now succeed when the nulls
are masked by parent-level nulls. This is a correctness fix, not a
behavior change — the previous behavior was incorrect per the Arrow
columnar format specification.
* GitHub Issue: #50515
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
