Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11502#discussion_r55115257
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -744,7 +750,7 @@ private[spark] class BlockManager(
level: StorageLevel,
tellMaster: Boolean = true,
effectiveStorageLevel: Option[StorageLevel] = None,
- keepReadLock: Boolean = false): Option[PutResult] = {
+ keepReadLock: Boolean = false): Option[Option[Iterator[Any]]] = {
--- End diff --
`doPut()` can be called to write either bytes or an iterator of values
(since `doPut` is used in the implementation of both `putIterator` and
`putBytes`).
Imagine that `putBytes()` fails because there was not enough space to store
the block in memory and it could not be dropped to disk. In this case, the
`put()` call has failed but we can't return an iterator.
Thus, we need to make a case class so that we can convey the three separate
outcomes (put of bytes failed, put of iterator failed, and put succeeded or
block was already present).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
