Steve Carlin has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/22405 )
Change subject: IMPALA-13576: Fix filter placement in the plan and related changes. ...................................................................... IMPALA-13576: Calcite Planner: handle filter above sort. This commit fixes the problem if there is a Filter RelNode above a Sort RelNode. IMPALA-13578: Calcite planner: Removed empty sort keys Added a rule to remove empty sort keys. The query select '1' from alltypestiny order by 1 was failing because the expression didn't exist. There was a bug for the remove sort keys rule in 1.36 so Calcite is being upgraded in this change as well. IMPALA-13579: Calcite planner: Allow limit clause in subquery Small parser fix to allow the limit clause in a subquery. IMPALA-13577: Push Filter RelNode through Project RelNode While this is an optimization, it also has the benefit that less code is needed when converting logical Calcite nodes to Impala PlanNodes. There is no concept of a Filter PlanNode in Impala. Instead, the filter condition always attaches itself to an underlying plan node. Therefore it is beneficial to have the filter node as close to a real plan node as we can. Pushing the filter node through the project node prevents extra processing that would be needed to change the filter condition to use the proper input ref. For instance, if we have a simple query like: select id from tbl where condition = 3; If the filter were on top of the project, the project would have to project up the filter column and the column may need to be renumbered. By pushing it through, the RelNode tree becomes simpler. This is somewhat of a bogus example since Calcite already places the filter underneath the Project when writing out the RelNode tree, but there are cases similar to this involving more complex queries. There is one caveat to this. In order to get around the Calcite issue of literal types using char (Impala uses string), a bogus passthrough Project is created as part of the CoerceNodes step. The filter pushthrough is done before the CoerceNodes step by design, since the passthrough Project needs to be next to the Values RelNode when doing the Calcite to PlanNode conversion. This is the only time we allow a Filter to be above a Project, which is checked via an assert. Change-Id: Id075d8516f1fcff4e6402c2ab4b4992a174c8151 --- M java/calcite-planner/pom.xml M java/calcite-planner/src/main/codegen/templates/Parser.jj M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaProjectRel.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaSortRel.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteOptimizer.java 5 files changed, 16 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/22405/3 -- To view, visit http://gerrit.cloudera.org:8080/22405 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id075d8516f1fcff4e6402c2ab4b4992a174c8151 Gerrit-Change-Number: 22405 Gerrit-PatchSet: 3 Gerrit-Owner: Steve Carlin <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]>
