Github user liyezhang556520 commented on the pull request:
https://github.com/apache/spark/pull/3629#issuecomment-69282553
Hi @andrewor14 , I think @suyanNone 's explain is correct. What @suyanNone
wants to say that if we just remove the [memory
release](https://github.com/apache/spark/blob/d345ebebd554ac3faa4e870bd7800ed02e89da58/core/src/main/scala/org/apache/spark/storage/MemoryStore.scala#L290)
in unrollSafely, then the memory that marked as unrolled will be never
released, even after the the corresponding block is actually put into the
memory. You can check in *ensureFreeMemory* method, the `atcualMemory` is
calculated as `val actualFreeMemory = freeMemory - currentUnrollMemory`. In
which, the `currentUnrollMemory` will be never decreased. That is the double
counting of unrollmemory in @suyanNone 's explanation (one copy already put in
memory, one copy unrolled in unrollMemory).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
