kosiew opened a new pull request, #20836:
URL: https://github.com/apache/datafusion/pull/20836
## Which issue does this PR close?
* Part of #20164.
## Rationale for this change
The physical planner currently lowers logical `CAST` expressions using only
the target `DataType`. That drops logical field semantics such as metadata,
field name, and intended nullability. As a result, casts to extension-annotated
fields or other metadata-bearing fields either lose information during lowering
or are rejected by planner-side validation that exists only because the
physical cast path is type-only.
This change makes cast lowering field-aware so the produced `CastExpr`
preserves the logical target field semantics. It also keeps validation for
supported casts in place while removing the planner restriction that blocked
metadata-bearing cast targets.
## What changes are included in this PR?
* Added `cast_with_target_field_and_options` in
`physical-expr/src/expressions/cast.rs` to build a `CastExpr` from a full
target `FieldRef` instead of only a `DataType`.
* Updated the existing `cast_with_options` helper to delegate to the new
field-aware path.
* Changed logical-to-physical cast lowering in
`physical-expr/src/planner.rs` to pass the logical target field through
unchanged.
* Removed the planner-level rejection for `Expr::Cast` targets with
non-empty metadata.
* Kept cast compatibility validation intact for both standard casts and
struct-to-struct casts.
* Exported the new helper within the expressions module for planner use.
* Added planner-focused unit tests covering:
* preservation of extension metadata on cast target fields
* preservation of target field semantics, including name, metadata, and
nullability
* same-type casts still producing a `CastExpr` when field semantics must
be preserved
* no regression for standard casts without metadata
* continued rejection of unsupported `TryCast` cases involving extension
metadata
## Are these changes tested?
Yes.
This PR adds targeted unit coverage in the physical planner tests to verify:
* cast lowering preserves target field metadata
* cast lowering preserves nullability intent from the logical field
* same-type casts retain the logical target field rather than collapsing
back to the input column expression
* standard non-metadata casts continue to behave as expected
* unsupported `TryCast` behavior still fails with the expected validation
signal
These tests are focused on planner behavior and help guard against
regressions in metadata and field-semantics propagation.
## Are there any user-facing changes?
Yes, but they are behavioral rather than API-breaking.
Logical `CAST` expressions that carry field metadata or nullability intent
will now preserve those semantics in the lowered physical expression instead of
dropping them or being rejected during planning. This is especially relevant
for extension-type metadata and other metadata-aware downstream consumers.
No public API changes are intended.
## LLM-generated code disclosure
This PR includes LLM-generated code and comments. All LLM-generated content
has been manually reviewed and tested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
