Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14289
Change subject: IMPALA-8647: fix round-to-zero in planner estimates ...................................................................... IMPALA-8647: fix round-to-zero in planner estimates Zero estimates in query plans can be quite dangerous (for causing bad plans) because they propagate up the plan tree, causing many more estimates to be zero. E.g. if the estimate for the right side of the join becomes 0, but the left side of the join has 1 trillion rows, the planner will think that the join produces zero rows and make further decisions accordingly. This could be catastrophic if the right side of the join produces even a single row. This adds some helper functions that avoid rounding to zero when multiplying selectivities and uses them in places where selectivities were computed. The scan nodes already bumped up the estimates anyway, so this doesn't change cardinality estimates or plans, but SelectNode and AggregationNode will not round to zero after this change. I looked at using Math.ceil() instead of Math.round(), but it changed estimates slightly in a lot of plans, which didn't seem worth the hassle. Testing: Added targeted cardinality tests for SelectNode and AggregationNode. Updated and sanity checked planner tests where estimates changed. Change-Id: I148e9f1aede0a1e99b875b0e6af534f4bccf49b7 --- M fe/src/main/java/org/apache/impala/planner/AggregationNode.java M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/SelectNode.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-nested.test M testdata/workloads/functional-planner/queries/PlannerTest/views.test 12 files changed, 642 insertions(+), 632 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/14289/1 -- To view, visit http://gerrit.cloudera.org:8080/14289 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I148e9f1aede0a1e99b875b0e6af534f4bccf49b7 Gerrit-Change-Number: 14289 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>
