bert-beyondloops opened a new pull request, #25330:
URL: https://github.com/apache/datafusion/pull/25330
- Closes #25329
## Rationale for this change
`CommonSubexprEliminate` extracts a repeated sub-expression into its own
`__common_expr_N` projection column so it's computed once instead of once per
occurrence.
Separately, `extract_leaf_expressions` (the leaf-pushdown optimizer) tries
to push cheap, `MoveTowardsLeafNodes`-classified expressions down closer to
the base table scan.
The bug: once CSE rewrites a repeated expression to a bare `Column`
reference to `__common_expr_N`, that reference is indistinguishable from a
genuine cheap base-table column at the `ExpressionPlacement` level. If it's
wrapped in something like `get_field(__common_expr_N, 'name')`, the
leaf-pushdown pass can classify the whole thing as `MoveTowardsLeafNodes` and
push it down — which requires re-deriving `__common_expr_N`'s original
(potentially expensive) defining expression at the new, lower location. That
silently duplicates the exact computation CSE had just deduplicated, defeating
the purpose of CSE for any expression that also gets leaf-pushed.
## What changes are included in this PR?
- Add `references_cse_common_expr` in `common_subexpr_eliminate.rs`, which
checks whether an expression contains a column reference to a CSE-synthesized
`__common_expr_N` alias.
- In `extract_leaf_expressions.rs`, short-circuit `routing_extract` for
`MoveTowardsLeafNodes` expressions that reference a CSE common-expr column:
leave them exactly where CSE placed them instead of extracting/pushing
them down.
## What is the testing strategy for this PR?
Added a new `sqllogictest` case in
`datafusion/sqllogictest/test_files/cse.slt`
## Are there any user-facing changes?
No public API changes. This is a bug fix affecting query plans: some
queries combining CSE-eligible sub-expressions with leaf-pushdown-eligible
wrapping expressions (e.g. `get_field`) will now correctly evaluate the common
sub-expression once instead of duplicating it, which may change generated
`EXPLAIN` output but not query results (results were already correct; only
redundant computation is removed).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
