Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22235 )
Change subject: IMPALA-13622: Fix negative cardinality bug in AggregationNode.java ...................................................................... IMPALA-13622: Fix negative cardinality bug in AggregationNode.java An incomplete COMPUTE STATS during data loading reveal a bug in AggregationNode.java where estimateNumGroups() can return value less than -1. This patch fix the bug by implementing PlanNode.smallestValidCardinality() and MathUtil.saturatingMultiplyCardinalities(). Both function validates that the function arguments are valid cardinality number. smallestValidCardinality() correctly compares two cardinality numbers and return the smallest and valid one. It generalizes and replaces static function PlanNode.capCardinalityAtLimit(). saturatingMultiplyCardinalities() adds validation and normalization over MathUtil.saturatingMultiply(). Reorder logic of tuple-based estimation from IMPALA-13405 such that negative estimate is handled properly. Testing: - Added more preconditions in AgggregationNode.java. - Added CardinalityTest.testSmallestValidCardinality and MathUtilTest.testSaturatingMultiplyCardinality. - Added test in resource-requirements.test that will consistently fail without this fix. - Pass testResourceRequirement. Change-Id: Ib862a010b2094daa2cbdd5d555e46443009672ad Reviewed-on: http://gerrit.cloudera.org:8080/22235 Reviewed-by: Impala Public Jenkins <[email protected]> Reviewed-by: Jason Fehr <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/planner/AggregationNode.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/SortNode.java M fe/src/main/java/org/apache/impala/util/MathUtil.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M fe/src/test/java/org/apache/impala/util/MathUtilTest.java M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test 7 files changed, 614 insertions(+), 44 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified Jason Fehr: Looks good to me, but someone else must approve -- To view, visit http://gerrit.cloudera.org:8080/22235 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ib862a010b2094daa2cbdd5d555e46443009672ad Gerrit-Change-Number: 22235 Gerrit-PatchSet: 11 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Pranav Lodha <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]>
