GitHub user NarineK opened a pull request:
https://github.com/apache/spark/pull/13760
[SPARK-16012][SparkR] GapplyCollect - applies a R function to each group
similar to gapply and collects the result back to R data.frame
## What changes were proposed in this pull request?
gapplyCollect() does gapply() on a SparkDataFrame and collect the result
back to R. Compared to gapply() + collect(), gapplyCollect() offers performance
optimization as well as programming convenience, as no schema is needed to be
provided.
This is similar to dapplyCollect().
## How was this patch tested?
Added test cases for gapplyCollect similar to dapplyCollect
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NarineK/spark gapplyCollect
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13760.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13760
----
commit ea31820c9501d1f8cba96bc7f8e0fab04e9af0a2
Author: Narine Kokhlikyan <[email protected]>
Date: 2016-06-17T10:51:15Z
initial version of gapplyCollect
commit f8e54dc265ad0eb66a26508bc5221606a9652e22
Author: Narine Kokhlikyan <[email protected]>
Date: 2016-06-17T11:00:05Z
merged with master
commit 591c4804764cdce67d22ce52ec38c74f246e738b
Author: Narine Kokhlikyan <[email protected]>
Date: 2016-06-17T11:03:04Z
revert .gitignore
commit 37b633afdff46374d983d204f13c05c769c7f40e
Author: Narine Kokhlikyan <[email protected]>
Date: 2016-06-18T08:59:05Z
added test cases + improved the code
commit de5dbb0be0a3fcc42096a10470c543eaf7aa6d5c
Author: Narine Kokhlikyan <[email protected]>
Date: 2016-06-18T09:05:57Z
fixed test case
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
