Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/10170#issuecomment-163086853
Note to self RE: unroll fraction (just to be precise):
At a high-level, the StaticMemoryManager a request for unroll memory will
be able to obtain more memory if:
- there is free storage memory, or
- `currentUnrollMemory / maxStorageMemory` < `spark.memory.unrollFraction`.
An unroll request can only evict blocks if `finalUnrollMemory /
maxStorageMemory` < `spark.memory.unrollFraction`, where `finalUnrollMemory` is
the amount of unroll memory that would be used after the eviction was performed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
