GitHub user woggle opened a pull request:
https://github.com/apache/spark/pull/5784
[SPARK-7214] Reserve space for unrolling even when MemoryStore nearly full
If making the initial reservation of space to unrolling a block fails,
attempt to drop blocks to make room before giving up.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/woggle/spark SPARK-7214
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5784.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5784
----
commit d6002d8d8d39e068b4ae5ce9efe920e06b6f72ef
Author: Charles Reiss <[email protected]>
Date: 2015-04-28T22:48:31Z
Test for unrolling when unrollMemoryThreshold is not available but would be
if blocks are dropped.
commit 6c57a97fd0ea2a93ee797060fcb540d05b7dcdc1
Author: Charles Reiss <[email protected]>
Date: 2015-04-29T05:37:32Z
Consistently call ensureFreeSpace() when reserving unroll memory.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
