Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/17683#discussion_r112416878
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala
---
@@ -516,19 +522,32 @@ private[spark] class BlockManagerInfo(
* They can be both larger than 0, when a block is dropped from
memory to disk.
* Therefore, a safe way to set BlockStatus is to set its info in
accurate modes. */
var blockStatus: BlockStatus = null
+
if (storageLevel.useMemory) {
blockStatus = BlockStatus(storageLevel, memSize = memSize,
diskSize = 0)
_blocks.put(blockId, blockStatus)
_remainingMem -= memSize
- logInfo("Added %s in memory on %s (size: %s, free: %s)".format(
- blockId, blockManagerId.hostPort, Utils.bytesToString(memSize),
- Utils.bytesToString(_remainingMem)))
+ if (blockExists) {
+ logInfo("Updated %s in memory on %s (current size: %s, original
size %s, free: %s)".format(
+ blockId, blockManagerId.hostPort, Utils.bytesToString(memSize),
+
Utils.bytesToString(originalMemSize),Utils.bytesToString(_remainingMem)))
+ } else {
+ logInfo("Added %s in memory on %s (size: %s, free: %s)".format(
+ blockId, blockManagerId.hostPort, Utils.bytesToString(memSize),
+ Utils.bytesToString(_remainingMem)))
+ }
}
if (storageLevel.useDisk) {
blockStatus = BlockStatus(storageLevel, memSize = 0, diskSize =
diskSize)
_blocks.put(blockId, blockStatus)
- logInfo("Added %s on disk on %s (size: %s)".format(
- blockId, blockManagerId.hostPort, Utils.bytesToString(diskSize)))
+ if (blockExists) {
+ logInfo("Updated %s on disk on %s (current size: %s, original
size %s)".format(
--- End diff --
Yes, you've got it. It's worth changing all of the related log messages
here, but not necessarily every one in the file. We are slowly updating to use
string interpolation as this code gets updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
