mkleen opened a new pull request, #25126: URL: https://github.com/apache/datafusion/pull/25126
feat: Simplify min(distinct x)/max(distinct x) to min(x)/max(x) ## Which issue does this PR close? - Relates to https://github.com/apache/datafusion/issues/24929 but does not close it, only optimizes the min/max case ## Rationale for this change The aggregation functions `min`/`max` are duplicate insensitive: - `min(DISTINCT x)` is identical `min(x)` - `min(DISTINCT x)` is identical `min(x)` Therefore `distinct` can be removed from min/max during expression simplification. This prevents that the optimizer rule `SingleDistinctToGroupBy` fires in the following examples which leads to a less efficient plan: ``` EXPLAIN FORMAT indent SELECT g, min(DISTINCT x) FROM t GROUP BY g; ``` Before this change: ``` Projection: t.g, min(alias1) AS min(DISTINCT t.x) Aggregate: groupBy=[[t.g]], aggr=[[min(alias1)]] Aggregate: groupBy=[[t.g, t.x AS alias1]], aggr=[[]] TableScan: t projection=[g, x] ``` after: ``` Aggregate: groupBy=[[t.g]], aggr=[[min(t.x) AS min(DISTINCT t.x)]] TableScan: t projection=[g, x] ``` The second plan is much more memory efficient: ## What changes are included in this PR? - Extend of min/max to simplify min(distinct x)/max(distint x) to min(x)/max(x) - Adoption of expression simplifier to only report a change if a simplication happend - Unalias the expression in SingleDistinctToGroupBy to make this optimization work in the rule - Tests ## What is the testing strategy for this PR? Existing tests pass; new slt added. ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
