Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r198656910
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -723,7 +770,9 @@ private[spark] class BlockManager(
}
if (data != null) {
- return Some(new ChunkedByteBuffer(data))
+ val chunkSize =
+ conf.getSizeAsBytes("spark.storage.memoryMapLimitForTests",
Int.MaxValue.toString).toInt
--- End diff --
Want to turn this into a config constant? I'm seeing it in a bunch of
places.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
