Hello Zoltan Borok-Nagy, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22897
to look at the new patch set (#4).
Change subject: IMPALA-14071: Refactor
MathUtil.saturatingMultiplyCardinalities()
......................................................................
IMPALA-14071: Refactor MathUtil.saturatingMultiplyCardinalities()
There are multiple ways to do cardinality multipication that also avoid
integer overflow. Some helper methods available are:
MathUtil.saturatingMultiplyCardinalities()
PlanNode.checkedMultiply()
LongMath.saturatingMultiply()
This patch intent to simplify things by unifying
MathUtil.saturatingMultiplyCardinalities() with
PlanNode.checkedMultiply(). The latter is maintained, while also adding
cardinality Preconditions check. Similar Preconditions checks are also
added into PlanNode.checkedAdd().
Harden cardinality calculation in several places by using checkedAdd()
and checkedMultiply() accordingly. Added sanity check
PlanNode.verifyCardinality() that is evaluated at the end of
PlanNode.computeStats(). This ensure that cardinality_ and
inputCardinality_ is always valid after PlanNode.computeStats().
Also fixed bug in ExchangeNode.estimateTotalQueueByteSize() that prevent
calculation against negative cardinality or negative num nodes.
Testing:
- Pass CardinalityTest and MathUtilTest.
Change-Id: I505ab11cfa1024feb4ceac4cffe9c3283be228ce
---
M fe/src/main/java/org/apache/impala/planner/AggregationNode.java
M fe/src/main/java/org/apache/impala/planner/ExchangeNode.java
M fe/src/main/java/org/apache/impala/planner/HBaseScanNode.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java
M fe/src/main/java/org/apache/impala/planner/NestedLoopJoinNode.java
M fe/src/main/java/org/apache/impala/planner/PlanNode.java
M fe/src/main/java/org/apache/impala/planner/ScanNode.java
M fe/src/main/java/org/apache/impala/util/MathUtil.java
M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java
M fe/src/test/java/org/apache/impala/util/MathUtilTest.java
M
testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M
testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test
M
testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-ddl-iceberg.test
M
testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-ddl-parquet.test
M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test
16 files changed, 137 insertions(+), 117 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/97/22897/4
--
To view, visit http://gerrit.cloudera.org:8080/22897
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I505ab11cfa1024feb4ceac4cffe9c3283be228ce
Gerrit-Change-Number: 22897
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
Gerrit-Reviewer: Wenzhe Zhou <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
