viirya commented on a change in pull request #23701: [SPARK-26741][SQL] Allow
using aggregate expressions in ORDER BY clause
URL: https://github.com/apache/spark/pull/23701#discussion_r253015981
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1679,7 +1694,31 @@ class Analyzer(
}
def containsAggregate(condition: Expression): Boolean = {
- condition.find(_.isInstanceOf[AggregateExpression]).isDefined
+ condition.find(e =>
+ e.isInstanceOf[AggregateExpression] ||
+ e.isInstanceOf[GroupingID] || e.isInstanceOf[Grouping]).isDefined
+ }
+
+ private def pushDownMissingAttrs(
+ missingAttrs: Seq[NamedExpression], plan: LogicalPlan): LogicalPlan = {
+ // Missing attributes can be unresolved attributes or resolved
attributes which are not in
+ // the output attributes of the plan.
+ plan match {
+ case p: Project =>
+ // Recursively pushing down expressions on the child of current plan.
+ val newChild = pushDownMissingAttrs(missingAttrs, p.child)
+ Project(p.projectList ++ missingAttrs.map(_.toAttribute), newChild)
+
+ case a @ Aggregate(_, aggExprs, _) =>
+ a.copy(aggregateExpressions = aggExprs ++ missingAttrs)
+
+ // For other operators (eg. Filter), push down recursively
+ case n: UnaryNode =>
Review comment:
It sounds too ambiguous to have `UnaryNode` here because it doesn't define
clearly what this operator will do. If there is an `UnaryNode` changing outputs
from its child, unexpected result will get. It sounds much safer to write
`Filter` here, except for that we are 100% sure that `UnaryNode` won't change
its child's output.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
