Github user mikeringenburg commented on the pull request:
https://github.com/apache/spark/pull/5403#issuecomment-93057465
We'd be very interesting in trying this out - we've been pointing
spark.local.dir to a ram disk when we need better shuffle performance, but have
run into a number of issues with that model. We'd love to try out this
in-memory shuffle to see if it helps.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
