adriangb opened a new pull request, #25412:
URL: https://github.com/apache/datafusion/pull/25412
## Which issue does this PR close?
None filed.
## Rationale for this change
This query fails to plan on `main`:
```sql
create table t(v int, s struct<a int>, env varchar) as values (1, {a: 10},
'prod'), (2, {a: 20}, 'dev');
with samples as (
select v, s, env from t
),
expanded as (
select v, s, env from samples
union all
select v, s, env from samples where 1 = 2
)
select env, sum(s['a']) from expanded group by env;
```
```
Optimizer rule 'push_down_leaf_projections' failed
caused by
Schema error: Schema contains qualified field name samples.env and
unqualified
field name env which would be ambiguous
```
`enable_leaf_expression_pushdown` is on by default, so this is a plain
planning
failure for a valid statement. A `union all` whose one side is provably
empty is
what produces the shape, and query generators emit that: a dashboard panel
adds a
second branch behind a comparison of two constants and the panel stops
working.
## What changes are included in this PR?
`build_extraction_projection_impl` merges an extraction projection into the
projection below it, and then adds the pass-through columns the merged
projection
does not already carry. It compared the columns it was about to add against
the
projection's own expressions without putting the two in the same space.
A projection can list bare column names over a qualified input. Removing the
empty
side of the union leaves exactly that. The comparison then misses, the
column is
added a second time under its bare name, and `Projection::try_new` rejects a
schema
that holds `samples.env` and a bare `env` together.
This PR resolves both sides against the input schema before the comparison,
and
pushes the column under the name the input gives it. A name the input schema
holds
more than once resolves to nothing, because no single spelling is correct
there.
## Are these changes tested?
Yes. The statement above is added to `struct.slt`. It fails on `main` with
the
error above and passes with this change.
`cargo test --workspace --exclude datafusion-sqllogictest` passes: 126
suites,
12479 tests, 0 failures. The sqllogictest suite passes too: 520 files, 0
failures. `cargo clippy -p datafusion-optimizer --all-targets` and
`cargo fmt --all -- --check` are clean.
## Are there any user-facing changes?
A statement of this shape plans instead of failing. No API change.
## Note on overlapping work
https://github.com/apache/datafusion/pull/25388 (draft) edits the same arm of
`build_extraction_projection_impl`, for a different problem: duplicated
evaluation of `KeepInPlace` expressions,
https://github.com/apache/datafusion/issues/25329.
It does not touch the qualifier comparison this PR changes, so the two are
independent in behaviour, but whichever lands second will need a small
rebase at
the tail of the `columns_needed` loop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
