kosiew opened a new issue, #24506:
URL: https://github.com/apache/datafusion/issues/24506
# Avoid casting unreachable child values in nested List casts
## Problem
`cast_list_column` and `cast_list_view_column` preserve parent offsets/sizes
and recursively cast the full backing child array. A sliced or limited List can
therefore fail because an unreachable child value cannot be cast.
Example:
```sql
CREATE TABLE lt AS SELECT * FROM (VALUES
(1, [struct('1')]), (2, [struct('2')]), (3, [struct('bad')])
) AS t(i, l);
SELECT arrow_cast(l, 'List(Struct("c0": Int32))') FROM lt LIMIT 2;
-- currently fails while casting the unreachable `bad` value
```
## Invariant / desired behavior
Recursive nested casts must process only child values reachable from
visible, non-null parent rows. Hidden backing children must not produce cast
errors.
## Proposed direction
Establish one consistent child-selection strategy for List, LargeList, and
ListView casts, analogous to Map entry compaction. Preserve visible offsets,
nulls, and schema while avoiding recursive casts of unreachable children.
## Scope
### In
- List, LargeList, and ListView recursive cast paths in
`datafusion/common/src/nested_struct.rs`
- Slice and null-parent regression coverage
### Out
- Map behavior (already compacts entries)
- FixedSizeList behavior, unless the shared solution naturally applies
## Acceptance criteria
- [ ] A cast succeeds when invalid nested child data is reachable only
through sliced-out or null parent rows.
- [ ] Visible invalid child data still returns a cast error.
- [ ] Output offsets, nulls, and data type are preserved.
## Tests / verification
- Unit regressions for List, LargeList, and ListView sliced/null-parent
inputs.
- Regression based on the SQL example above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
