GitHub user aarondav opened a pull request:
https://github.com/apache/spark/pull/874
Introduce SchemaRDD#aggregate() for simple aggregations
```scala
rdd.aggregate(Sum('val))
```
is just shorthand for
```scala
rdd.groupBy()(Sum('val))
```
but seems be more natural than doing a groupBy with no grouping expressions
when you really just want an aggregation over all rows.
Did not add a JavaSchemaRDD or Python API, as these seem to be lacking
several other methods like groupBy() already -- leaving that cleanup for future
patches.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aarondav/spark schemardd
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/874.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #874
----
commit db6afe2f6f055ac2b86004c83aa16e326e78b433
Author: Aaron Davidson <[email protected]>
Date: 2014-05-25T20:53:28Z
Introduce SchemaRDD#aggregate() for simple aggregations
rdd.aggregate(Sum('val)) is just shorthand for
rdd.groupBy()(Sum('val)), but seems be more natural than
doing a groupBy with no grouping expressions when you
really just want an aggregation over all rows.
Did not add a JavaSchemaRDD or Python API, as these seem to
be lacking in several other methods like groupBy() already --
leaving that cleanup for future patches.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
