Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24592
to look at the new patch set (#3).
Change subject: IMPALA-15189: Support HBO for SortNode cardinality
......................................................................
IMPALA-15189: Support HBO for SortNode cardinality
A sort only reorders rows, so its output row count equals its child's
unless it applies a limit: TopN, Partitioned TopN, or a total sort
carrying a LIMIT. This extends HBO to support tracking and using
cardinality of such SortNodes.
The HBO key string of a SortNode consists of
- Type: TOTAL, PARTIAL, TOPN, PARTITIONED_TOPN
- IncludeTies: true/false for Top-N
- OrderKind: LEXICAL, ZORDER
- LIMIT, OFFSET for non-PARTITIONED_TOPN
- PerPartitionLimit and Partition Exprs for PARTITIONED_TOPN
- Order Exprs
- Child key
Skip local TopN since its cardinality usages assumes it's a per-instance
cardinality, but HBO tracks the summed cross-instance cardinality. Merge
TopN will skip its local TopN in constructing the child key string,
similar to final AggregationNode skips intermediate AggregationNodes.
This ensures the merge TopN's HBO key is identical to the single-node
plan's key.
Testing
- Added FE tests on HBO key strings
- Added e2e tests
Assisted-by: Opus 4.8 (Claude Code)
Change-Id: Ib829887a91593bee124d56e661e714575fe3be97
---
M fe/src/main/java/org/apache/impala/planner/AggregationNode.java
M fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java
M fe/src/main/java/org/apache/impala/planner/ExprCanonicalizer.java
M fe/src/main/java/org/apache/impala/planner/JoinNode.java
M fe/src/main/java/org/apache/impala/planner/SortNode.java
M fe/src/test/java/org/apache/impala/planner/HboKeyStringTest.java
A
testdata/workloads/functional-query/queries/QueryTest/hbo-agg-on-topn-sort.test
M tests/query_test/test_hbo.py
8 files changed, 445 insertions(+), 25 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/24592/3
--
To view, visit http://gerrit.cloudera.org:8080/24592
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib829887a91593bee124d56e661e714575fe3be97
Gerrit-Change-Number: 24592
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
