maropu commented on a change in pull request #27198:
[SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of
AggregateExpression
URL: https://github.com/apache/spark/pull/27198#discussion_r366166932
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala
##########
@@ -149,10 +149,24 @@ case class AggregateExpression(
case PartialMerge => "merge_"
case Final | Complete => ""
}
- prefix + aggregateFunction.toAggString(isDistinct)
+ val aggFuncStr = prefix + aggregateFunction.toAggString(isDistinct)
+ mode match {
+ case Partial | Complete if filter.isDefined =>
Review comment:
Doing so shows a filter in a second aggregate in physical plans like this;
```
== Physical Plan ==
HashAggregate(keys=[k#0], functions=[sum(cast(v#1 as bigint)) filter ((v#1 >
3))], output=[k#0, sum(v) FILTER ((v > 3))#44L])
+- Exchange hashpartitioning(k#0, 200), true, [id=#154]
+- HashAggregate(keys=[k#0], functions=[partial_sum(cast(v#1 as bigint))
filter ((v#1 > 3))], output=[k#0, sum#48L])
+- *(1) ColumnarToRow
+- FileScan parquet default.t[k#0,v#1] Batched: true, DataFilters:
[], Format: Parquet, Location:
InMemoryFileIndex[file:/Users/maropu/Repositories/spark/spark-master/spark-warehouse/t],
PartitionFilters: [], PushedFilters: [], ReadSchema: struct<k:int,v:int>
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
