Baymine opened a new issue, #66044: URL: https://github.com/apache/doris/issues/66044
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version master / 4.0 ### What's Wrong? The `MultiDistinct` strategy keeps a per-group hash set of every distinct value on a single node. For a group-by query whose distinct argument is near-unique (or whose statistics are unknown and could be near-unique), one node can end up holding a whole group's value set and OOM, e.g.: ```sql select count(distinct payment_id), ... from t group by merchant_id ``` The no-group-by branch of `DistinctAggStrategySelector.shouldUseMultiDistinct` already guards against this by checking distinct-argument NDV and routing unknown-statistics cases to the CTE split strategy. The group-by branch does not: it neither checks distinct-argument NDV, nor treats unknown distinct-argument statistics as risky, and on unknown group-by statistics it unconditionally chooses `MultiDistinct`. ### What You Expected? For a group-by query whose distinct argument is near-unique relative to input rows, or whose distinct argument (or a slot it reads) has unknown statistics, the planner should route to the CTE split strategy (which redistributes on the distinct key) to avoid the single-node OOM — mirroring the no-group-by branch. ### How to Reproduce? Run a group-by aggregation with one or more `count(distinct ...)` over a near-unique column (or a column wrapped in `if(cond, col, null)` where `col` is unanalyzed), and observe that `MultiDistinct` is chosen and can OOM under a low-cardinality group-by key. ### Anything Else? Fix (group-by branch of `shouldUseMultiDistinct`), matching the no-group-by branch: 1. `hasHighNdvDistinctArgument`: force CTE split when any distinct argument's estimated NDV >= `row * MID_CARDINALITY_THRESHOLD`. 2. `hasUnknownNdvDistinctArgument`: force CTE split when a distinct argument's estimate is unknown, or any input slot it reads has absent/unknown statistics (the per-slot scan is required because `ExpressionEstimation.visitIf` fabricates a non-unknown NDV for `if(cond, col, null)` even when `col` is unanalyzed). 3. When group-by statistics are unknown (reached only when every distinct argument is confirmed low-NDV), prefer CTE split for a single group-by key and keep `MultiDistinct` for >= 2 keys. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
