oversk7 opened a new issue, #54907:
URL: https://github.com/apache/arrow/issues/54907
### Describe the bug, including details regarding any error messages,
version, and platform.
`SparseUnionArray::Make` uses `type_ids.offset()` as the offset of the new
union array, but it checks the children's length against `type_ids.length()`
only. As a result, there is no way to build a valid sparse union from a sliced
`type_ids` array:
- Children of length `type_ids.length()` are accepted. However, the
resulting array is invalid: union slot `i` refers to child position
`type_ids.offset() + i`, which these children do not have.
- Children of length `type_ids.offset() + type_ids.length()`, which the
resulting array would need, are rejected.
Reproducer (pyarrow 26.0.0 from PyPI, Python 3.12, Windows 11).
`UnionArray.from_sparse` calls `SparseUnionArray::Make` and then `Validate()`:
```python
import pyarrow as pa
type_ids = pa.array([0, 1, 0], pa.int8()).slice(1) # offset 1, length 2
for n in (2, 3):
children = [pa.array(range(n), pa.int64()), pa.array(["s"] * n)]
try:
pa.UnionArray.from_sparse(type_ids, children)
print(n, "accepted")
except Exception as e:
print(n, e)
# 2 Sparse union child array #0 has length smaller than expected for union
array (2 < 3)
# 3 Sparse UnionArray must have len(child) == len(type_ids) for all children
```
Without the `Validate()` call that pyarrow adds, the first case silently
produces an array that maps its slots to the wrong child positions.
Code:
[array_nested.cc](https://github.com/apache/arrow/blob/373d64c02cb23663b63c5853099fca43a46c39fc/cpp/src/arrow/array/array_nested.cc#L1396-L1428).
One possible fix is to slice the type ids buffer and build the array with
offset 0. Children would then keep corresponding to the logical positions of
`type_ids`, as the existing error message ("len(child) == len(type_ids)")
suggests. Another option is to require `len(child) == type_ids.offset() +
type_ids.length()`.
This was found with an AI-assisted analysis of the C++ code against the
columnar format specification. The reproducer above was run against pyarrow
26.0.0.
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
