Hello Quanlong Huang, Steve Carlin, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24693
to look at the new patch set (#6).
Change subject: IMPALA-15236: Expose HBO match provenance
......................................................................
IMPALA-15236: Expose HBO match provenance
The plan says "(from HBO)" when a cardinality comes from a historical
run, but not which key matched or what the planner would have estimated
on its own. Investigating a surprising estimate means turning on debug
logging and running the query again.
Record the match on the PlanNode and add a details line next to the
cardinality naming the canonicalization strategy that matched, the
opaque hash key, and the estimate the planner had computed. The query
profile always carries it: that is the artifact left behind once the
query is gone, and the one profile tools read. EXPLAIN carries it only
under the new ENABLE_EXPLAIN_HBO query option, so the text that tooling
parses does not grow a line per plan node by default.
Not every match deserves the same trust, and that belongs where the
number is rather than behind an option. IGNORE_PARTITION_CONSTANTS drops
the constants from partition equality predicates and assumes all
partitions look alike, so the run it matched may have been recorded for
a different partition than the one being planned. Such a match now reads
"(from HBO, partition constants ignored)". An EXPR_REWRITE match keeps
its constants and describes the query being planned, so it stays
"(from HBO)" and the default EXPLAIN text is unchanged there.
Reset the match metadata before each lookup so a later miss cannot show
stale provenance left by an earlier pass over the same node.
Testing:
- HistoricalStatsTest
- TestHBO::test_matching_provenance
Change-Id: I6d2deaaf78a2a41353454ba634a247d8c69825bf
Assisted-by: claude-opus-5 (Claude Code)
---
M be/src/service/query-options.cc
M be/src/service/query-options.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M fe/src/main/java/org/apache/impala/planner/CanonicalizationStrategy.java
M fe/src/main/java/org/apache/impala/planner/PlanNode.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/HistoricalStats.java
M fe/src/test/java/org/apache/impala/service/HistoricalStatsTest.java
M testdata/workloads/functional-query/queries/QueryTest/hbo-multiple-scans.test
M
testdata/workloads/functional-query/queries/QueryTest/hbo-single-scan-partitioned-stats.test
M tests/query_test/test_hbo.py
13 files changed, 245 insertions(+), 37 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/24693/6
--
To view, visit http://gerrit.cloudera.org:8080/24693
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6d2deaaf78a2a41353454ba634a247d8c69825bf
Gerrit-Change-Number: 24693
Gerrit-PatchSet: 6
Gerrit-Owner: Aleksandr Efimov <[email protected]>
Gerrit-Reviewer: Aleksandr Efimov <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Steve Carlin <[email protected]>
