Zach Amsden has uploaded a new patch set (#11). Change subject: IMPALA-5003: Constant propagation in scan nodes and inline views ......................................................................
IMPALA-5003: Constant propagation in scan nodes and inline views When conjuncts are pushed into table refs and inline views, they can be considered for constant progagation within that node. In certain cases, we might end up with a FALSE conditional and now we can convert ScanNodes to EmptySet nodes when that occurs. I also added an inequality collation phase which is now partially tested and will combine conjuncts such as a < k1, a < k2 into a < min(k1, k2), as well as detect equivalence from a >= k, a <= k, and determine conflicting bounds requirements to be false. This could be expanded to do analysis against other slotrefs in the future, but this should probably be saved for another diff. Testing: Expanded the test cases for the planner to achieve constant propagation. Added Kudu, datasource, Hdfs and HBase tests to validate we can create EmptySetNodes. Some manual testing for inequality conjuncts but nothing formal yet. Query: explain select * from functional_hbase.widetable_250_cols a where a.int_col1 > 1 and a.int_col1 <= 20 and a.int_col1 < 50 and a.int_col1 > 2 +------------------------------------------------------------------- | Explain String +------------------------------------------------------------------- | Estimated Per-Host Requirements: Memory=1.00GB VCores=1 | PLAN-ROOT SINK | | | 01:EXCHANGE [UNPARTITIONED] | | | 00:SCAN HBASE [functional_hbase.widetable_250_cols a] | predicates: a.int_col1 <= 20, a.int_col1 > 2 +------------------------------------------------------------------- Fetched 10 row(s) in 0.08s Change-Id: I79750a8edb945effee2a519fa3b8192b77042cb4 --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/Expr.java M fe/src/main/java/org/apache/impala/analysis/SelectList.java M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/main/java/org/apache/impala/planner/ValueRange.java M fe/src/main/java/org/apache/impala/rewrite/ExprRewriter.java M fe/src/main/java/org/apache/impala/rewrite/NormalizeBinaryPredicatesRule.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test M testdata/workloads/functional-planner/queries/PlannerTest/conjunct-ordering.test A testdata/workloads/functional-planner/queries/PlannerTest/constant-propagation.test M testdata/workloads/functional-planner/queries/PlannerTest/data-source-tables.test M testdata/workloads/functional-planner/queries/PlannerTest/hdfs.test M testdata/workloads/functional-planner/queries/PlannerTest/joins.test M testdata/workloads/functional-planner/queries/PlannerTest/kudu.test M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test 19 files changed, 644 insertions(+), 117 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/6389/11 -- To view, visit http://gerrit.cloudera.org:8080/6389 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I79750a8edb945effee2a519fa3b8192b77042cb4 Gerrit-PatchSet: 11 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Zach Amsden <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-Reviewer: Zach Amsden <[email protected]>
