GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/8083
[SPARK-9702] Use Exchange to perform shuffle for Repartition operator
This patch allows `Repartition` to support UnsafeRows. This is accomplished
by implementing the logical `Repartition` operator in terms of `Exchange` and a
new `RandomPartitioning`.
Closes #8030.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark SPARK-9702
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8083.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8083
----
commit 0e0a8d899a8f93fbf778534372a9389ce06781c4
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-08-07T09:03:53Z
Use Exchange to perform shuffle for Repartition operator.
commit c0c724a255860d9f0503511b911a00c660bd1d88
Author: Josh Rosen <[email protected]>
Date: 2015-08-10T22:29:20Z
Support UnsafeRow in Repartition.
commit 0b8c624cb5d768a7df8dc186f16a0e8d5fa653c0
Author: Josh Rosen <[email protected]>
Date: 2015-08-10T22:41:00Z
Randomly partition rows in Repartition.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
