Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/791#issuecomment-55337768
Let me raise the same question here that I raised in #2134. If my
understanding is correct, by the time `ensureFreeSpace` returns we aren't
guaranteed to have actually freed up the space requested, but we return
"success" to the caller anyway. Doesn't this cause a potential race condition
where we put a new block faster than the old block is dropped? And this is
likely if we put the new block into memory but the old block onto disk. I
haven't followed the details of the above conversation, but the consequence of
this condition could be an OOM if we're already at the edge of the total memory.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
