englefly opened a new pull request, #70332: URL: https://github.com/apache/doris/pull/70332
### Background Nereids query profiles retain a physical plan to display estimated and actual row counts. They also print scan column statistics in the profile appendix. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix Problem Summary: Profiles currently retain the planner's physical plan and scan nodes, allowing their GroupExpression/Group references and lazy property suppliers to keep optimizer state alive after planning. Rendering actual row counts also writes into Statistics shared with the planner, and repeated scans print every column statistic repeatedly. Shallow-copy the physical plan for profile display and capture only known operative column statistics once per qualified table and column. ### Details - **Change map:** 233 added / 18 removed production lines and 330 added / 10 removed test lines; no generated files or environment changes. Production changes cover the profile capture path, a plan copier, and small copy overrides for operators with embedded plans. - **Frequency:** Queries that collect profiles and retain physical plans; repeated table references and wide tables make the column-statistics duplication more visible. - **Root cause:** Profile stores planner-owned plan/scan references. Selected plan nodes retain GroupExpression references, logical properties contain plan-capturing suppliers, and column statistics are printed without operative/unknown filtering or table-column deduplication. - **Why existing tests missed it:** Existing profile tests check rendering and retry lifecycle, including shared-plan actual-row mutations, but do not check object reachability or operative-column deduplication. - **Fix:** Rebuild plan nodes with shared expressions/table metadata, captured output properties, independent Statistics, and scalar Group IDs. Reconnect runtime filters to copied builders/targets in their original order. Copy embedded scans and lazy-materialization relation maps as well. Store the column appendix as text so it retains no scan references. Deduplication keeps the first known statistic for each qualified table and column; an earlier unknown statistic does not suppress a later known one. - **Other paths and branches:** Remove the StmtExecutor assignment that would overwrite the copied plan. Cover OLAP/file/TVF lazy scans, lazy materialization, storage-layer aggregates, SQL-cache plans, and retry reset paths. This PR changes profile retention; HBO's independent plan cache is outside this change. - **Behavior change:** The column appendix omits unknown and non-operative columns and repeated table-column entries. Physical-plan node IDs, Group IDs, runtime-filter order, and live actual-row display are preserved. Query execution and estimates are unchanged. ### Testing - `./run-fe-ut.sh --run 'org.apache.doris.common.profile.ProfilePlanTest,org.apache.doris.common.profile.ProfileTest,org.apache.doris.qe.StmtExecutorTest'`: 78 tests passed locally (7 copier/statistics tests, 16 profile tests, 55 executor/retry tests). - `mvn -f fe/pom.xml checkstyle:check -pl fe-core`: passed, zero violations. - `git diff --cached --check`: passed. - The full BE/FE build and regression suites are left to CI; `run buildall` and `/review` will be requested on the Draft PR. ### Notes for Reviewers - **Goal and scope:** Changes are confined to profile plan retention/display and the copying support needed for embedded plans. Expressions and table metadata remain shared. - **Concurrency:** No new threads or locks. Copy state belongs to one capture operation; existing profile synchronization remains in place. Rendering mutates only profile-owned Statistics. - **Lifecycle:** Reachability tests inspect suppliers, mutable state, embedded plans, and runtime-filter links without relying on GC timing. Retry tests check that failed-attempt statistics and actual rows are cleared. - **Parallel paths and conditions:** Both plan setters now preserve isolation; custom copy paths handle embedded plans. Missing statistics and unknown statistics are intentionally omitted. - **Compatibility and configuration:** No new configuration, FE/BE fields, transaction/data-write changes, or persistent metadata schema changes. Existing profile spill/read behavior is exercised by the retry tests; the appendix text becomes more compact. - **Coverage and observability:** Tests cover cloning, RF ordering, embedded OLAP/file/TVF scans, relation maps, SQL cache, known/unknown operative columns, aliases, distinct databases, and retries. Plan identifiers remain available for diagnosis. No metrics are needed for this display change. - **Performance:** Copy nodes and mutable Statistics while sharing expressions, columns and metadata. Format each selected table-column statistic once. No measured performance claim. ### Release note Query profiles retain less optimizer state and print only known statistics for operative columns, once per qualified table and column. ### Check List (For Author) - Test: Unit tests and FE Checkstyle (see Testing) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Partially verified: 78 local FE tests and Checkstyle passed; full build/regression CI pending. Submitted as Draft as requested. ### AI Disclosure - Tool: OpenAI Codex (GPT-6) - Generated: Implementation, unit tests, commit message, and PR description. - Human review: Not yet reviewed by the author; this is a Draft PR for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
