[
https://issues.apache.org/jira/browse/IMPALA-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17951540#comment-17951540
]
ASF subversion and git services commented on IMPALA-14070:
----------------------------------------------------------
Commit f7cf4f8446642b7c50c54c5cba977a4fcdf9a6e3 in impala's branch
refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f7cf4f844 ]
IMPALA-14070: Use checkedMultiply in SortNode.java
maxRowsInHeaps calculation may overflow because it use simple
multiplication. This patch fix the bug by calculating it using
checkedMultiply(). A broader refactoring will be done by IMPALA-14071.
Testing:
Add ee tests TestTopNHighNdv that exercise the issue.
Change-Id: Ic6712b94f4704fd8016829b2538b1be22baaf2f7
Reviewed-on: http://gerrit.cloudera.org:8080/22896
Reviewed-by: Abhishek Rawat <[email protected]>
Reviewed-by: Wenzhe Zhou <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> SortNode's computeStats can calculate invalid cardinality
> ---------------------------------------------------------
>
> Key: IMPALA-14070
> URL: https://issues.apache.org/jira/browse/IMPALA-14070
> Project: IMPALA
> Issue Type: Bug
> Reporter: Zoltán Borók-Nagy
> Assignee: Riza Suminto
> Priority: Major
>
> The problem is with this code fragment in SortNode.computeStats:
> {noformat}
> long partNdv = numPartitionExprs_ == 0 ? 1 :
> Expr.getNumDistinctValues(partExprs);
> if (partNdv >= 0) {
> long maxRowsInHeaps = partNdv * getPerPartitionLimit(); <====
> maxRowsInHeaps can overflow and can become less than -1
> if (cardinality_ < 0 || cardinality_ > maxRowsInHeaps) {
> cardinality_ = maxRowsInHeaps;
> }
> }
> {noformat}
> If maxRowsInHeaps overflows and becomes less than -1 then the cardinality
> will be invalid and we will hit this precondition check later in
> PlanNode.smallestValidCardinality:
> {noformat}
> protected static long smallestValidCardinality(long cardinality1, long
> cardinality2) {
> Preconditions.checkArgument(
> cardinality1 >= -1, "cardinality1 is invalid: %s", cardinality1);
> Preconditions.checkArgument(
> cardinality2 >= -1, "cardinality2 is invalid: %s", cardinality2);
> ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
