englefly opened a new pull request, #70636:
URL: https://github.com/apache/doris/pull/70636

   ### Background
   
   HBO identifies a plan subtree with GroupStructInfo and uses its structural 
fingerprint to look up pinned and learned row counts. The current builder 
follows Memo groups. Profile physical-plan copies deliberately detach their 
GroupExpression references so that retained profiles do not retain the 
optimizer.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: #69322 (prerequisite, cherry-picked), #70332 (prerequisite, 
cherry-picked)
   PR Type: Enhancement
   
   Problem Summary:
   
   Add an on-demand GroupStructInfo.fromPhysicalPlan entry point for selected 
subtrees of a detached physical plan. It shares node-description construction, 
expression normalization, rendering and fingerprint encoding with the Memo 
builder. This enables a later learned-HBO collector to select candidates after 
execution without computing every fingerprint during planning or retaining Memo.
   
   ### Details
   
   - **Change map:** The final enhancement commit changes one production file 
(239 added / 40 removed lines) and adds 400 lines in two test files. The branch 
also includes all prerequisite commits from #69322 and #70332; review the final 
commit for this enhancement.
   - **Before and after:** Previously, structInfo required a GroupExpression or 
a group-ID-to-Group map. The new entry point follows physical children directly 
and needs neither. Physical distributions and sorts are transparent, inner join 
chains are flattened and ordered, computed projection definitions are resolved, 
and aggregate execution wrappers are replaced with their original function 
identity.
   - **Stage semantics:** Matching two-stage non-distinct aggregates and merge 
TopN implementations are normalized to their logical identity. SplitLimit runs 
before Memo construction, so its two levels remain part of the structural 
identity. Partial roots, DISTINCT aggregation phases, unmatched 
aggregation/TopN stages and shared physical inputs return explicit unsupported 
reasons.
   - **Configs and defaults:** No new configuration or production caller. 
Existing HBO collection, publishing and Profile behavior remain unchanged. The 
caller must use the execution session context when building/materializing 
session-dependent expressions; scan metadata is read when the description is 
built.
   - **Impact on existing users:** Existing Memo fingerprints and serialized 
key formats are preserved. This does not introduce level 0 collection or 
replace the existing learned-HBO publishing path.
   
   ### Testing
   
   - `./run-fe-ut.sh --run` with 22 explicitly named stats/Profile test 
classes: 234 existing tests passed. Two fixture failures in the new classes 
were corrected (clone the whole RF-connected plan; use analyzed Union output 
slots).
   - `MAVEN_OPTS="-Dmaven.main.skip=true" ./run-fe-ut.sh --run 
'org.apache.doris.nereids.stats.PhysicalPlanStructInfoTest,org.apache.doris.nereids.stats.PhysicalPlanStructInfoPlannerTest'`:
 all 11 new tests passed after those corrections. Production classes were 
reused from the preceding full compilation; only test sources changed. Combined 
result: 245 passing tests across 22 classes.
   - The new tests compare canonical descriptions, shapes and fingerprints in 
both literal modes; cover scan/filter/project, hash and nested-loop joins, join 
reordering, single/two-stage aggregation and HAVING, nested aggregates, 
distributed bounds, Union mappings and unsupported cases. Real post-processed 
plans are compared after Memo release, and a pinned entry keyed from a clone is 
applied during replanning.
   - `mvn -f fe/pom.xml checkstyle:check -pl fe-core`: passed, zero violations.
   - `git diff --cached --check`: passed.
   - Full BE/FE packaging and regression suites were not run locally; `run 
buildall` and `/review` will be requested on this Draft PR.
   
   ### Notes for Reviewers
   
   - Clone the complete physical plan before selecting candidate subtrees: 
runtime filters can connect a subtree to other nodes. The planner test uses a 
full clone and verifies that Memo has been released before describing 
candidates.
   - Concurrency and lifecycle: traversal state is local to one call; no 
threads, locks or global caches are introduced. The physical visited set is 
allocated only for the new entry point.
   - Errors and observability: unsupported structures return an invalid 
description with a reason, following the existing structInfo contract; 
invariant failures are not silently converted into approximate keys.
   - Memory: the new traversal never resolves a Group or group ID and retains 
no Memo references from detached input plans.
   - Correctness and compatibility: golden keys, both literal modes, 
physical/Memo equivalence and pinned lookup are checked. Execution, transaction 
visibility and persistent metadata are unchanged. BE nullable handling is not 
applicable.
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Behavior changed: No (new internal entry point only)
   - Does this need documentation: No
   - Pre-submission status: Partially verified: local FE tests and Checkstyle; 
full build and regression CI pending. Submitted as Draft as requested.
   
   ### AI Disclosure
   
   - Tool: OpenAI Codex
   - Generated: Implementation, tests, commit message and PR description.
   - Human review: Not yet reviewed by the author; submitted as Draft.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to