Github user wypoon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23058#discussion_r235238066
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -693,9 +693,9 @@ private[spark] class BlockManager(
*/
private def getRemoteValues[T: ClassTag](blockId: BlockId):
Option[BlockResult] = {
val ct = implicitly[ClassTag[T]]
- getRemoteBytes(blockId).map { data =>
+ getRemoteManagedBuffer(blockId).map { data =>
val values =
- serializerManager.dataDeserializeStream(blockId,
data.toInputStream(dispose = true))(ct)
+ serializerManager.dataDeserializeStream(blockId,
data.createInputStream())(ct)
--- End diff --
I added an assert in `getRemoteManagedBuffer` itself. There are 3 places
where a value can be returned in the method, and only one of them is nonEmpty.
I added the assert there.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
