GitHub user dilipbiswal opened a pull request:
https://github.com/apache/spark/pull/22809
[SPARK-19851][SQL] Add support for EVERY and ANY (SOME) aggregates
## What changes were proposed in this pull request?
Implements Every, Some, Any aggregates in SQL. These new aggregate
expressions are analyzed in normal way and rewritten to equivalent existing
aggregate expressions in the optimizer.
Every(x) => Min(x) where x is boolean.
Some(x) => Max(x) where x is boolean.
Any is a synonym for Some.
SQL
```
explain extended select every(v) from test_agg group by k;
```
Plan :
```
== Parsed Logical Plan ==
'Aggregate ['k], [unresolvedalias('every('v), None)]
+- 'UnresolvedRelation `test_agg`
== Analyzed Logical Plan ==
every(v): boolean
Aggregate [k#0], [every(v#1) AS every(v)#5]
+- SubqueryAlias `test_agg`
+- Project [k#0, v#1]
+- SubqueryAlias `test_agg`
+- LocalRelation [k#0, v#1]
== Optimized Logical Plan ==
Aggregate [k#0], [min(v#1) AS every(v)#5]
+- LocalRelation [k#0, v#1]
== Physical Plan ==
*(2) HashAggregate(keys=[k#0], functions=[min(v#1)], output=[every(v)#5])
+- Exchange hashpartitioning(k#0, 200)
+- *(1) HashAggregate(keys=[k#0], functions=[partial_min(v#1)],
output=[k#0, min#7])
+- LocalTableScan [k#0, v#1]
Time taken: 0.512 seconds, Fetched 1 row(s)
```
## How was this patch tested?
Added tests in SQLQueryTestSuite, DataframeAggregateSuite
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dilipbiswal/spark SPARK-19851-specific-rewrite
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22809.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22809
----
commit e6c5c84ea189b1fdbac8408594c880950b6b7398
Author: Dilip Biswal <dbiswal@...>
Date: 2018-10-16T06:23:41Z
tests
commit b793d06cb937db300c78e4eb4cd143c385419e57
Author: Dilip Biswal <dbiswal@...>
Date: 2018-10-23T16:43:04Z
Code changes
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
