[
https://issues.apache.org/jira/browse/IMPALA-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith resolved IMPALA-15278.
------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> Incorrect cardinality with OFFSET in MERGING-EXCHANGE and TOP-N
> ---------------------------------------------------------------
>
> Key: IMPALA-15278
> URL: https://issues.apache.org/jira/browse/IMPALA-15278
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> In ExchangeNode.computeStats(), the cardinality is first capped by the limit
> and then decreased by offset:
> {code:java}
> cardinality_ = capCardinalityAtLimit(children_.get(0).getCardinality());
> // Apply the offset correction if there's a valid cardinality
> if (cardinality_ > -1) cardinality_ = Math.max(0, cardinality_ - offset_);
> {code}
> https://github.com/apache/impala/blob/c47f8cce53e3262bde67f881f9591be93b831fe2/fe/src/main/java/org/apache/impala/planner/ExchangeNode.java#L155-L157
> This is incorrect. Cardinality should subtract the offset first and then cap
> at the limit.
> Take the following query as an example:
> {code:sql}
> select id, int_col from functional.alltypes order by id limit 1000 offset
> 6200{code}
> It returns 1000 rows but the estimated cardinality is 0 due to this bug:
> {noformat}
> 02:MERGING-EXCHANGE [UNPARTITIONED]
> | offset: 6200
> | order by: id ASC
> | limit: 1000
> | mem-estimate=36.00KB mem-reservation=0B thread-reservation=0
> | tuple-ids=1 row-size=8B cardinality=0
> | in pipelines: 01(GETNEXT){noformat}
> TOP-N SortNode has another issue that the offset is not used in computing
> cardinality.
> {code:java}
> if (isTypeTopN() && includeTies_) {
> cardinality_ =
> MathUtil.smallestValidCardinality(getChild(0).cardinality_,
> limitWithTies_);
> } else {
> cardinality_ = capCardinalityAtLimit(getChild(0).cardinality_);
> }{code}
> https://github.com/apache/impala/blob/c47f8cce53e3262bde67f881f9591be93b831fe2/fe/src/main/java/org/apache/impala/planner/SortNode.java#L306-L311
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
