Hello Yida Wu, Jason Fehr, Pranav Lodha, Michael Smith, Csaba Ringhofer, Impala
Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22235
to look at the new patch set (#6).
Change subject: IMPALA-13622: Fix negative cardinality bug in
AggregationNode.java
......................................................................
IMPALA-13622: Fix negative cardinality bug in AggregationNode.java
An incomplete COMPUTE STATS during data loading reveal a bug in
AggregationNode.java where estimateNumGroups() can return value less
than -1.
This patch fix the bug by implementing
PlanNode.smallestValidCardinality() and
MathUtil.saturatingMultiplyCardinalities(). Both function validates that
the function arguments are valid cardinality number.
smallestValidCardinality() correctly compares two cardinality numbers
and return the smallest and valid one. It generalize and replace static
function PlanNode.capCardinalityAtLimit().
saturatingMultiplyCardinalities() adds validation and normalization over
MathUtil.saturatingMultiply().
Reorder logic of tuple-based estimation from IMPALA-13405 such that
negative estimate is handled properly.
Testing:
- Added more preconditions in AgggregationNode.java.
- Added CardinalityTest.testSmallestValidCardinality and
MathUtilTest.testSaturatingMultiplyCardinality.
- Added test in resource-requirements.test that will consistently fail
without this fix.
- Pass testResourceRequirement.
Change-Id: Ib862a010b2094daa2cbdd5d555e46443009672ad
---
M fe/src/main/java/org/apache/impala/planner/AggregationNode.java
M fe/src/main/java/org/apache/impala/planner/PlanNode.java
M fe/src/main/java/org/apache/impala/planner/SortNode.java
M fe/src/main/java/org/apache/impala/util/MathUtil.java
M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java
M fe/src/test/java/org/apache/impala/util/MathUtilTest.java
M
testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
7 files changed, 617 insertions(+), 44 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/22235/6
--
To view, visit http://gerrit.cloudera.org:8080/22235
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib862a010b2094daa2cbdd5d555e46443009672ad
Gerrit-Change-Number: 22235
Gerrit-PatchSet: 6
Gerrit-Owner: Riza Suminto <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Jason Fehr <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Pranav Lodha <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
Gerrit-Reviewer: Yida Wu <[email protected]>
