Michael Smith has uploaded this change for review. (
http://gerrit.cloudera.org:8080/22091
Change subject: IMPALA-13531: Calcite CTE frontend
......................................................................
IMPALA-13531: Calcite CTE frontend
Support identifying CTEs in the Calcite planner for single-node plans.
CTE detection is based on HIVE-28259.
Adds new logical nodes Sequence, CTEProducer, and CTEConsumer, matching
constructs outlined in "Optimization of Common Table Expressions in MPP
Database Systems" by El-Helw et al. Implements them with physical
SequenceNode, CTEBufferNode, and CTEScanNode.
Adds cte_threshold query option to control when CTE replacement occurs.
CTE replacement will only occur when number of occurences exceeds the
threshold. Defaults to 1 (so minimum 2 occurences needed). Set to -1 to
disable CTE planning completely.
Some basic tests look like
set num_nodes=1;
with v1 as (select n_name, n_nationkey
from tpch.nation where n_regionkey = 0)
select v1.n_name from v1, v1 v1_next
where v1.n_nationkey + 1 = v1_next.n_nationkey;
create table emps
(empid int, deptno int, name string, salary decimal(8, 2));
insert into emps values (1, 1, "Phil", 45000.0),
(2, 1, "Maggie", 55000.0), (3, 2, "Alice", 120000.0);
set num_nodes=1;
SELECT name FROM emps e WHERE salary > 50000
UNION SELECT name FROM emps e WHERE salary > 50000;
Change-Id: Id0840c0859d2fe25628d799a18d302cee1eb36e8
---
M common/thrift/ImpalaService.thrift
M common/thrift/PlanNodes.thrift
M common/thrift/Query.thrift
A fe/src/main/java/org/apache/impala/planner/CTEBufferNode.java
A fe/src/main/java/org/apache/impala/planner/CTEScanNode.java
A fe/src/main/java/org/apache/impala/planner/SequenceNode.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/coercenodes/CoerceNodes.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/cte/Exception.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/cte/IdentitySuggester.java
A java/calcite-planner/src/main/java/org/apache/impala/calcite/cte/Registry.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/cte/Suggester.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/cte/SuggesterFactory.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaCTEConsumer.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaCTEProducer.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaPlanRel.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaSequence.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/CTERuleConfig.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/CommonRelSubExprRegisterRule.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/RemoveInfrequentCTERule.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteTable.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteOptimizer.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ExecRequestCreator.java
25 files changed, 1,393 insertions(+), 15 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/22091/1
--
To view, visit http://gerrit.cloudera.org:8080/22091
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id0840c0859d2fe25628d799a18d302cee1eb36e8
Gerrit-Change-Number: 22091
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <[email protected]>
