Ruchirtripathi opened a new pull request, #24147:
URL: https://github.com/apache/datafusion/pull/24147
## Which issue does this PR close?
- Closes #24139 .
## Rationale for this change
The `scalar_subquery_to_join` optimizer rule fails with a `Schema error:
No field named __always_true` when a CTE (`WITH ... AS (...)`) or a
`SubqueryAlias` is used inside a correlated scalar subquery.
This happens because during the decorrelation phase
(`PullUpCorrelatedExpr`), DataFusion adds a synthetic `__always_true` column to
the inner
`Aggregate` or `Projection` to help resolve the "count bug" for outer
joins on empty batches. However, when the subquery was wrapped in a
`SubqueryAlias`, the alias node was returning its cached schema instead of
recomputing it to expose the newly added `__always_true` column from its
child, causing the optimizer to fail downstream.
## What changes are included in this PR?
- Updated the `f_up` rewrite logic in
`datafusion/optimizer/src/decorrelate.rs` for `LogicalPlan::SubqueryAlias`.
- The rule now correctly rebuilds the alias node using
`SubqueryAlias::try_new(alias.input, alias.alias)` when its underlying input
changes. This
ensures the output schema of the `SubqueryAlias` is updated and correctly
propagates pulled-up columns (like `__always_true`) upward.
## Are these changes tested?
- Yes, this fix was verified locally against the reproducer query
provided in the issue. The query now correctly evaluates to a join and
successfully
outputs results without failing during optimization.
## Are there any user-facing changes?
- No, this is purely an internal optimizer fix.
Just copy everything inside the block above and submit your Pull Request!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
