Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/12822#discussion_r61710851
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
---
@@ -104,6 +101,27 @@ trait CheckAnalysis extends PredicateHelper {
failAnalysis(s"Window specification $s is not valid
because $m")
case None => w
}
+
+ case s @ ScalarSubquery(query, conditions, _) if
conditions.nonEmpty =>
+ // Make sure correlated scalar subqueries contain one row for
every outer row by
+ // enforcing that they are aggregates which contain exactly
one aggregate expressions.
+ // The analyzer has already checked that subquery contained
only one output column, and
+ // added all the grouping expressions to the aggregate.
+ def checkAggregate(a: Aggregate): Unit = {
+ val aggregates = a.expressions.flatMap(_.collect {
+ case a: AggregateExpression => a
+ })
+ if (aggregates.isEmpty) {
+ failAnalysis("The output of a correlated scalar subquery
must be aggregated")
+ }
+ }
+
+ query match {
+ case a: Aggregate => checkAggregate(a)
+ case Project(_, a: Aggregate) => checkAggregate(a)
+ case fail => failAnalysis(s"Correlated scalar subqueries
must be Aggregated: $fail")
--- End diff --
Can it have an Filter on top of Aggregate (HAVING clause)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
