waynexia opened a new pull request #792:
URL: https://github.com/apache/arrow-datafusion/pull/792
# Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate changelogs for our releases. You can
link an issue to this PR using the GitHub syntax. For example, `Closes #123`
indicates that this PR will close issue #123.
-->
Resolve #566.
# Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in
the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your
changes and offer better suggestions for fixes.
-->
This pull request trying to implement the common subexpression eliminate
optimization. The current implementation only supports detecting & eliminating
common subexpressions under one logical plan, not cross plans.
This optimizes is consists of two parts: detecting and replacing. It first
scans all expressions we are going to rewrite (all expressions under one plan
here), generates identifiers for each `Expr` node for later comparison. Then
for the same group of expressions, we extract common expressions, put them into
a `Projection` plan, and replace the original expression with a column `Expr`.
This implementation doesn't cover the situation of eliminating the whole
plan's common subexpression. This requires considering not only the expression
itself, but also sharing data with different plans or some plan will modify the
data (e.g. filter) or other aspects I haven't realize. In the beginning I plan
to make it cross plans but it seems better to me to do them in another PR.
# What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is
sometimes worth providing a summary of the individual changes in this PR.
-->
Add optimize rule `CommonSubexprEliminate`.
# Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
I think there is, as the optimization rule will rewrite input SQL.
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
# Status
This is still a work in progress. I have only covered part of plans and am
trying to make it work with the entire execution procedure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
