Repository: spark Updated Branches: refs/heads/master 5ad5e3486 -> dd96fcda0
Updated the comment for SPARK-2162. A follow up on #1103 @andrewor14 Author: Reynold Xin <[email protected]> Closes #1117 from rxin/SPARK-2162 and squashes the following commits: a4231de [Reynold Xin] Updated the comment for SPARK-2162. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/dd96fcda Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/dd96fcda Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/dd96fcda Branch: refs/heads/master Commit: dd96fcda0145810785b67f847f98b04ff7f0d7c3 Parents: 5ad5e34 Author: Reynold Xin <[email protected]> Authored: Wed Jun 18 12:48:58 2014 -0700 Committer: Reynold Xin <[email protected]> Committed: Wed Jun 18 12:48:58 2014 -0700 ---------------------------------------------------------------------- .../scala/org/apache/spark/storage/BlockManager.scala | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/dd96fcda/core/src/main/scala/org/apache/spark/storage/BlockManager.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/org/apache/spark/storage/BlockManager.scala b/core/src/main/scala/org/apache/spark/storage/BlockManager.scala index 373987c..d2f7baf 100644 --- a/core/src/main/scala/org/apache/spark/storage/BlockManager.scala +++ b/core/src/main/scala/org/apache/spark/storage/BlockManager.scala @@ -363,11 +363,13 @@ private[spark] class BlockManager( val info = blockInfo.get(blockId).orNull if (info != null) { info.synchronized { - // Double check to make sure the block is still there, since removeBlock - // method also synchronizes on BlockInfo object, so the block might have - // been removed when we actually come here. + // Double check to make sure the block is still there. There is a small chance that the + // block has been removed by removeBlock (which also synchronizes on the blockInfo object). + // Note that this only checks metadata tracking. If user intentionally deleted the block + // on disk or from off heap storage without using removeBlock, this conditional check will + // still pass but eventually we will get an exception because we can't find the block. if (blockInfo.get(blockId).isEmpty) { - logDebug(s"Block $blockId had been removed") + logWarning(s"Block $blockId had been removed") return None }
