Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/9344#issuecomment-152802034
Looking a bit more closely, it looks like all existing implementations of
`MemoryConsumer.spill()` will only end up reporting the size of Tugnsten pages
that are spilled. If Tungsten pages are allocated in-heap, then it makes sense
to try to spill in response to requests heap memory. If we're in off-heap mode,
though, then it doesn't make sense to try to spill when we're running short in
on-heap memory; in that mode, we should only spill in response to failed
off-heap memory requests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
