Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/156#issuecomment-38353582
Some time in future (not in this PR), we would want to add support for
disk-backed support for this feature : in case a partition is too large to fit
into memory. There is code already in spark which should make this reasonably
easy to add.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
