maropu commented on a change in pull request #31913:
URL: https://github.com/apache/spark/pull/31913#discussion_r600167101
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
##########
@@ -4140,6 +4140,20 @@ class SQLQuerySuite extends QueryTest with
SharedSparkSession with AdaptiveSpark
}
}
}
+
+ test("SPARK-34581: Don't optimize out grouping expressions from aggregate
expressions") {
+ withTempView("t") {
+ Seq[Integer](null, 1, 2, 3, null).toDF("id").createOrReplaceTempView("t")
+
+ val df = spark.sql(
Review comment:
If this is a test to check if a query work correctly or not, could you
move this test into `SQLQueryTestSuite`?
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
##########
@@ -66,6 +66,34 @@ object RewriteNonCorrelatedExists extends Rule[LogicalPlan] {
}
}
+/**
+ * Wrap complex grouping expression in aggregate expressions without aggregate
function into
Review comment:
What does "complex" means in this context?
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
##########
@@ -66,6 +66,34 @@ object RewriteNonCorrelatedExists extends Rule[LogicalPlan] {
}
}
+/**
+ * Wrap complex grouping expression in aggregate expressions without aggregate
function into
+ * `GroupingExpression` nodes so as to avoid further optimizations between the
expression and its
+ * parent.
+ *
+ * This is required as further optimizations could change the grouping
expression and so make the
+ * aggregate expression invalid.
Review comment:
Could you add a simple query example to explain about the case above?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
