Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21753#discussion_r202211733
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -586,12 +581,17 @@ class Analyzer(
}
}
- private def isAggregateExpression(expr: Expression): Boolean = {
- expr match {
- case Alias(e, _) => isAggregateExpression(e)
- case AggregateExpression(_, _, _, _) => true
- case _ => false
- }
+ // Support any aggregate expression that can appear in an Aggregate
plan except Pandas UDF.
+ // TODO: Support Pandas UDF.
+ private def checkValidAggregateExpression(expr: Expression): Unit =
expr match {
+ case _: AggregateExpression => // OK and leave the argument check to
CheckAnalysis.
+ case expr: PythonUDF if PythonUDF.isGroupedAggPandasUDF(expr) =>
--- End diff --
I created a JIRA for this support
https://issues.apache.org/jira/browse/SPARK-24796
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
