GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/9344
[SPARK-11389][WIP][CORE] Add support for off-heap memory to MemoryManager
In order to lay the groundwork for proper off-heap memory support in SQL /
Tungsten, we need to extend our MemoryManager to perform bookkeeping for
off-heap memory.
**Real PR description to follow shortly**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark offheap-memory-accounting
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/9344.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #9344
----
commit 64b7c8549e8b277bf5524f4bc59608440cbba066
Author: Josh Rosen <[email protected]>
Date: 2015-10-28T07:02:57Z
Factor cross-task memory arbitration into own component.
commit 8396ed638cee5aaae64d000d53f4cb0c8db768b3
Author: Josh Rosen <[email protected]>
Date: 2015-10-28T22:23:55Z
Hacky WIP approach at supporting fixed limit on off-heap execution memory.
commit f3f44fed63399992c7604a9db5b6d7810d57e286
Author: Josh Rosen <[email protected]>
Date: 2015-10-29T00:49:39Z
MemoryManager internals refactoring (WIP)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
