srowen commented on issue #23497: [SPARK-26578][CORE] Synchronize putBytes's memory allocation and putting block on memoryManager URL: https://github.com/apache/spark/pull/23497#issuecomment-453540126 Yeah @kiszk that's a good point. However none of the `entries.synchronized` blocks acquire the `memoryStore` lock; `evictBlocksToFreeSpace` locks a block in the BlockManager though. In that case, it has both the `memoryStore` and `entries` locks (in that order). It's also a good question: is this a problem? the issue I was worried about here is `evictBlocksToFreeSpace`. Is this a potential problem: 1. `putBytes` allocates memory (but hasn't added to `entries`) 1. `evictBlocksToFreeSpace` executes and calculates entries to evict and evicts entries 1. `putBytes` adds an entry, causing there to be not enough free memory I don't know this well enough to know whether that actually happens or is an issue, but is what jumped out at me. The 'allocation' method locks the `memoryStore` the whole time while the 'eviction' method doesn't.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
