dayan02dev opened a new pull request, #12365:
URL: https://github.com/apache/arrow-rs/pull/12365
# Which issue does this PR close?
- Closes #11608
# Rationale for this change
Since 60.1.0, `take` with a null index on a `StructArray` or
`FixedSizeListArray` fails when a child is run-end encoded with a non-nullable
values field ("Cannot take null indices from a RunEndEncoded array with a
non-nullable values field"), or is a union with no nullable field. 60.0.0
returned `Ok` for the same input. #10994 added these checks for top-level REE
and union arrays. The struct arm of `take_impl` and the slow path of
`take_fixed_size_list` still pass the null indices on to every child, so the
checks now fire there too. The parent's own validity already marks those rows
null, and the child values under them are unspecified. Jefffrey called this
"overly strict" on #10994 and suggested it as a separate issue.
# What changes are included in this PR?
- Struct and fixed-size list children go through a new `take_masked_child`.
A non-empty child whose type cannot store a null is taken with index 0 at the
null rows, the same way sparse union `take` already treats the children that
don't hold the null (`take_sparse_union_child`, unchanged).
- `data_type_can_represent_take_null` is split out of
`field_can_represent_take_null`, because a struct child's own nullability
doesn't matter here.
- All other children, empty children, and indices without nulls go to
`take_impl` exactly as before. An empty child has no value to use as a dummy,
so it keeps the existing error.
# Are these changes tested?
Yes. `test_take_null_indices_masked_by_parent` takes `[Some(1), None]` from
a struct and a fixed-size list over a non-nullable REE child and over a union
with no nullable field. It checks the parent nulls, `validate_full`, and the
valid row. It also pins the existing error for empty REE, union and no-field
union children. Without the fix it fails with the issue's REE error.
# Are there any significant user-facing changes?
No API changes. #11836 is the `MutableArrayData` counterpart and is not
touched here.
# AI usage disclosure
The patch, test and verification were produced with an AI coding agent
(Claude Code), including the base-versus-head checks for nested, sliced,
dense/sparse union and nullable-child cases and a struct `take` benchmark that
showed no change. I read the full diff and ran the new tests locally; they fail
on main and pass with this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]