Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19476#discussion_r145013046
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -653,15 +663,34 @@ private[spark] class BlockManager(
require(blockId != null, "BlockId is null")
var runningFailureCount = 0
var totalFailureCount = 0
- val locations = getLocations(blockId)
+
+ // Because all the remote blocks are registered in driver, so it is
not necessary to ask
+ // all the slave executors to get block status.
+ val locationAndStatus = master.getLocationsAndStatus(blockId)
+
+ val blockSize = locationAndStatus._2.map { status =>
+ // Disk size and mem size cannot co-exist, so it's ok to sum them
together to get block size.
+ status.diskSize + status.memSize
--- End diff --
The default value of diskSize or memSize can be 0 or -1 or even other
numbers, it's not guaranteed. Maybe we can use `Math.max(status.diskSize,
status.memSize)` instead?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
