Aleksandr Efimov has posted comments on this change. ( http://gerrit.cloudera.org:8080/24592 )
Change subject: IMPALA-15189: Support HBO for SortNode cardinality ...................................................................... Patch Set 6: (2 comments) http://gerrit.cloudera.org:8080/#/c/24592/6/fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java File fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java: http://gerrit.cloudera.org:8080/#/c/24592/6/fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java@1352 PS6, Line 1352: lowerTopN.setTopNMergeParent(upperTopN); > We can add lowerTopN.computeStats() here to clear the HBO cardinality. But Yes - the local Top-N in the new hbo-analytic-topn.test. It runs below the HASH exchange, so each instance keeps its own top 5 per group, and it emits at least as many rows as the merge Top-N returns. The golden gives it cardinality=50, which is the merged count, and the exchange between them takes its cardinality and its memory estimate from that number. It holds 50 only because it read HBO before the split, and nothing recomputes it afterwards. Should the comment here say that keeping it is deliberate? The isSortMergeInput() check in computeStats() skips HBO for a merge input, so the plan disagrees with it. http://gerrit.cloudera.org:8080/#/c/24592/6/fe/src/main/java/org/apache/impala/planner/SortNode.java File fe/src/main/java/org/apache/impala/planner/SortNode.java: http://gerrit.cloudera.org:8080/#/c/24592/6/fe/src/main/java/org/apache/impala/planner/SortNode.java@233 PS6, Line 233: return !(hasLimit() || isTypeTopN() || isPartitionedTopN()); > Nice catch! We should handle the case of OFFSET without LIMIT. Covered in PS7. originalOffset_ keeps isCardinalityPreserving() false after the offset moves to the exchange, and setMergeInfo() marks the local sort whether or not there is a limit. hbo-offset-only-sort.test covers the OFFSET-only plan. -- To view, visit http://gerrit.cloudera.org:8080/24592 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib829887a91593bee124d56e661e714575fe3be97 Gerrit-Change-Number: 24592 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Aleksandr Efimov <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Comment-Date: Wed, 02 Sep 2026 19:04:58 +0000 Gerrit-HasComments: Yes
