sdf-jkl opened a new pull request, #27141:
URL: https://github.com/apache/datafusion/pull/27141
## Which issue does this PR close?
Addresses #27023. Related to #14380.
Draft alternative to #27139 by @kumarUjjawal, retained for comparison and
potential contributions to that PR. This version includes UNION schema refresh
after parameter binding and compact SQL schema assertions. It is not ready to
merge.
## Rationale for this change
Before analysis, UNION reports the first input's type even when execution
uses a wider common type. For example, combining a microsecond UTC timestamp
with a nanosecond timestamp reports microseconds through `DataFrame::schema()`,
but returns nanoseconds during execution. CASE similarly reports its first
non-null THEN type, ignoring wider later branches.
## What changes are included in this PR?
Infer CASE and UNION output types using the existing coercion rules. CASE
considers every THEN and ELSE result; UNION combines input types by position or
name and refreshes its schema after parameter binding. UNION BY NAME
projections retain their actual input types so the analyzer can insert the
required casts.
Cast insertion remains in the analyzer, where expression and parameter types
have been resolved. This addresses the schema mismatch but does not fully
resolve #14380's request for fully coerced UNION inputs at construction.
Known limitation: dictionary coercion is incomplete here. For example,
`SELECT NULL AS x UNION ALL SELECT arrow_cast('a', 'Dictionary(Int32, Utf8)')
AS x` declares `Utf8` but executes as `Dictionary(Int32, Utf8)`. This changes
the executed type from the baseline and still leaves a schema mismatch. #27139
includes a correction for this path.
## What is the testing strategy for this PR?
Three `DESCRIBE SELECT` checks in `case.slt`, `union.slt`, and
`union_by_name.slt` assert schemas before analysis. They cover CASE branch
widening, timestamp precision/timezone, string representations, and name-based
UNION coercion with missing columns. All three fail before the fix and pass
afterward. Existing SLTs cover execution, including the NVL/UNION regression.
The full extended workspace suite passed before the test-only cleanup. After
replacing the added Rust tests with SLTs, all eight selected SQL files,
formatting, and all-features Clippy passed. A subsequent full-suite rerun was
interrupted when work switched to reviewing #27139.
## Are there any user-facing changes?
`DataFrame::schema()` and `DESCRIBE` report the inferred common types before
analysis. Incompatible CASE and UNION types fail during planning rather than
during analyzer coercion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
