Repository: spark Updated Branches: refs/heads/master 2f84a6866 -> e7bce9e18
[SPARK-17056][CORE] Fix a wrong assert regarding unroll memory in MemoryStore ## What changes were proposed in this pull request? There is an assert in MemoryStore's putIteratorAsValues method which is used to check if unroll memory is not released too much. This assert looks wrong. ## How was this patch tested? Jenkins tests. Author: Liang-Chi Hsieh <[email protected]> Closes #14642 from viirya/fix-unroll-memory. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e7bce9e1 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/e7bce9e1 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/e7bce9e1 Branch: refs/heads/master Commit: e7bce9e1876de6ee975ccc89351db58119674aef Parents: 2f84a68 Author: Liang-Chi Hsieh <[email protected]> Authored: Tue Sep 27 16:00:39 2016 -0700 Committer: Josh Rosen <[email protected]> Committed: Tue Sep 27 16:00:39 2016 -0700 ---------------------------------------------------------------------- .../main/scala/org/apache/spark/storage/memory/MemoryStore.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/e7bce9e1/core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala b/core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala index 205d469..095d324 100644 --- a/core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala +++ b/core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala @@ -273,7 +273,7 @@ private[spark] class MemoryStore( blockId, Utils.bytesToString(size), Utils.bytesToString(maxMemory - blocksMemoryUsed))) Right(size) } else { - assert(currentUnrollMemoryForThisTask >= currentUnrollMemoryForThisTask, + assert(currentUnrollMemoryForThisTask >= unrollMemoryUsedByThisBlock, "released too much unroll memory") Left(new PartiallyUnrolledIterator( this, --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
