Github user suyanNone commented on the pull request:
https://github.com/apache/spark/pull/3629#issuecomment-70077674
@andrewor14 @liyezhang556520
yes, because in current memorystore, it keep a `previousMemoryReserved` for
current Thread, that var is design to reserved the unroll memory for the block
can't put into `memorystore's entrys`.
```
// We ran out of space while unrolling the values for this block
logUnrollFailureMessage(blockId, vector.estimateSize())
Right(vector.iterator ++ values)
```
while I first read `previousMemoryReserved`, I also got confused.
as we know, put iterator value in memory block include:
broadcast(putSingle), RDD.
so there may have a possibility that current thread may use the
iterator-value of unroll-unsuccessful broadcast while unroll an RDD into
memoryStore, it may the reason why we need `previousMemoryReserved`.
right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
