Hello Joe McDonnell, Steve Carlin, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22094
to look at the new patch set (#10).
Change subject: IMPALA-13533: Calcite CTE backend
......................................................................
IMPALA-13533: Calcite CTE backend
Implement backend for CTEs in the Calcite planner for single-node plans.
CTE output is added to a BufferedTupleStream, then scanned concurrently.
Mimics UnionNode's MaterializeBatch for translating the CTE tuple to the
expected output tuple, with passthrough for cases where input and output
row layouts match.
Changes cte_threshold default to 1, so CTEs are enabled by default and
used when the cost-based planner identifies they would be helpful and at
least 2 instances of the CTE are detected.
Tested with TPC-DS queries (DecimalV2 version) and num_nodes=1 (via new
IMPALA_TEST_SINGLE_NODE option). All but 3 pass - Q18, Q22, and Q27 -
which is consistent with Calcite num_nodes=1 with no CTE support. Setup
start-impala-cluster.py --env_vars=USE_CALCITE_PLANNER=true \
--impalad_args=--default_query_options=use_calcite_planner=true
env IMPALA_TEST_SINGLE_NODE=1 impala-py.test \
tests/query_test/test_tpcds_queries.py::TestTpcdsDecimalV2Query
Change-Id: I48f16d495d4b37be97e6a913f0eb5b94d70e199a
---
M be/src/codegen/gen_ir_descriptions.py
M be/src/codegen/impala-ir.cc
M be/src/exec/CMakeLists.txt
A be/src/exec/cte-buffer-node.cc
A be/src/exec/cte-buffer-node.h
A be/src/exec/cte-scan-node-ir.cc
A be/src/exec/cte-scan-node.cc
A be/src/exec/cte-scan-node.h
M be/src/exec/exec-node.cc
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/scan-node.cc
A be/src/exec/sequence-node.cc
A be/src/exec/sequence-node.h
M be/src/runtime/buffered-tuple-stream.cc
M be/src/runtime/descriptors.cc
M be/src/runtime/runtime-state.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M tests/common/test_dimensions.py
M tests/util/parse_util.py
20 files changed, 746 insertions(+), 8 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/22094/10
--
To view, visit http://gerrit.cloudera.org:8080/22094
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48f16d495d4b37be97e6a913f0eb5b94d70e199a
Gerrit-Change-Number: 22094
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Steve Carlin <[email protected]>
