GitHub user massie opened a pull request:
https://github.com/apache/spark/pull/6423
Make internal Spark shuffle more customizable
o Updates the BlockStoreShuffleFetcher to allow callers to specify a custom
disk writer
o Updates the ShuffleBlockFetcherIterator to use allow callers to provide a
factory
object for creating custom iterators for reading over records in a
fetched block
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/massie/spark shuffle-api-cleanup
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/6423.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6423
----
commit becdc81a5a4305dfb4b1c2d03eeca232cff65de8
Author: Matt Massie <[email protected]>
Date: 2015-05-27T00:08:59Z
Make internal Spark shuffle more customizable
o Updates the BlockStoreShuffleFetcher to allow callers to specify a custom
disk writer
o Updated the ShuffleBlockFetcherIterator to use allow callers to provide a
factory
object for creating custom iterators for reading over records in a
fetched block
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
