sunchao opened a new issue, #24679:
URL: https://github.com/apache/datafusion/issues/24679
### Describe the bug
The Struct-cast narrowing added in #24125 does not distinguish casts
inserted by schema adaptation from explicit casts. It can also change the
result Field's inherited nullability and expose decimal conversion setup errors
that a whole-Struct cast skips for an all-null parent.
### To reproduce
Reproduced on `6e66a853b664b231737699d6704bdf721d44e92e` with Arrow 59.2.0:
1. Build a physical expression equivalent to `get_field(CAST(s AS STRUCT<x
INT, y INT>), 'x')`, with `s = {x: 1, y: 'bad'}`. Evaluating the original
explicit Struct cast raises an error for `y`; applying
`DefaultPhysicalExprAdapter::rewrite` can instead yield `1`. This is reproduced
directly at the physical-adapter boundary, not claimed for every SQL plan.
2. Adapt a required physical Struct parent to a nullable logical parent
whose selected child remains non-nullable. The rewritten field access must
remain nullable because the parent is nullable. Comparing only the child Fields
can lose that property, including through nested parents.
3. Adapt an all-null Struct containing an all-null Utf8 child to `Struct<x:
Decimal128(10, -1)>`. The original whole-Struct cast returns nulls; the
narrowed conversion fails with `Cannot cast string to decimal with negative
scale -1`. A decimal rescale from `(38, -38)` to `(38, 38)` has the same
setup-error problem. The string-to-decimal case also reproduces through an
actual Parquet filter with pushdown enabled.
### Expected behavior
Explicit casts retain their sibling conversions and errors. Rewritten
expressions preserve the complete logical return Field. Conversions with
setup-time errors keep the whole-Struct null shortcut, and already-pushed
predicates remain evaluable when a Struct cast is retained.
### Additional context
This is a focused follow-up to #24125 and #24530. It does not propose
generic encoded-array masking or claim to fix every existing null-parent
behavior of `get_field`.
AI assistance: Codex helped investigate the code, generate regression tests,
and write this report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
