This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new e9734962442 [SPARK-42259][SQL] ResolveGroupingAnalytics should take
care of Python UDAF
e9734962442 is described below
commit e97349624421afee07a68fda38027cfe7b796597
Author: Wenchen Fan <[email protected]>
AuthorDate: Wed Feb 1 17:36:14 2023 +0800
[SPARK-42259][SQL] ResolveGroupingAnalytics should take care of Python UDAF
This is a long-standing correctness issue with Python UDAF and grouping
analytics. The rule `ResolveGroupingAnalytics` should take care of Python UDAF
when matching aggregate expressions.
bug fix
Yes, the query result was wrong before
existing tests
Closes #39824 from cloud-fan/python.
Authored-by: Wenchen Fan <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 1219c8492376e038894111cd5d922229260482e7)
Signed-off-by: Wenchen Fan <[email protected]>
---
.../main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
index 4d1ea95d0e7..8c304d8a400 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
@@ -610,7 +610,7 @@ class Analyzer(override val catalogManager: CatalogManager)
// AggregateExpression should be computed on the unmodified value of
its argument
// expressions, so we should not replace any references to grouping
expression
// inside it.
- case e: AggregateExpression =>
+ case e if AggregateExpression.isAggregate(e) =>
aggsBuffer += e
e
case e if isPartOfAggregation(e) => e
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
