Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4827#discussion_r27091751
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -317,6 +337,29 @@ private[spark] class MemoryStore(blockManager:
BlockManager, maxMemory: Long)
value: Any,
size: Long,
deserialized: Boolean): ResultWithDroppedBlocks = {
+ tryToPut(blockId, () => value, size, deserialized)
+ }
+
+ /**
+ * Try to put in a set of values, if we can free up enough space. The
value should either be
+ * an Array if deserialized is true or a ByteBuffer otherwise. Its
(possibly estimated) size
+ * must also be passed by the caller.
+ *
+ * `value` will be lazily created. If it cannot be put into MemoryStore
or disk, `value` won't be
+ * created to avoid OOM since it may be a big ByteBuffer.
+ *
+ * Synchronize on `accountingLock` to ensure that all the put requests
and its associated block
+ * dropping is done by only on thread at a time. Otherwise while one
thread is dropping
+ * blocks to free memory for one block, another thread may use up the
freed space for
+ * another block.
+ *
+ * Return whether put was successful, along with the blocks dropped in
the process.
+ */
--- End diff --
Instead of duplicating the javadocs here I would just refer to the other
method. I can fix this when I merge don't worry.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
