Repository: spark Updated Branches: refs/heads/master d7b1fcf8f -> fba9cc846
[SPARK-21813][CORE] Modify TaskMemoryManager.MAXIMUM_PAGE_SIZE_BYTES comments ## What changes were proposed in this pull request? The variable "TaskMemoryManager.MAXIMUM_PAGE_SIZE_BYTES" comment error, It shouldn't be 2^32-1, should be 2^31-1, That means the maximum value of int. ## How was this patch tested? Existing test cases Author: he.qiao <[email protected]> Closes #19025 from Geek-He/08_23_comments. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fba9cc84 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/fba9cc84 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/fba9cc84 Branch: refs/heads/master Commit: fba9cc8466dccdcd1f6f372ea7962e7ae9e09be1 Parents: d7b1fcf Author: he.qiao <[email protected]> Authored: Tue Aug 29 23:44:27 2017 +0100 Committer: Sean Owen <[email protected]> Committed: Tue Aug 29 23:44:27 2017 +0100 ---------------------------------------------------------------------- .../main/java/org/apache/spark/memory/TaskMemoryManager.java | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/fba9cc84/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java ---------------------------------------------------------------------- diff --git a/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java b/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java index 761ba9d..0f1e902 100644 --- a/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java +++ b/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java @@ -53,8 +53,8 @@ import org.apache.spark.util.Utils; * retrieve the base object. * <p> * This allows us to address 8192 pages. In on-heap mode, the maximum page size is limited by the - * maximum size of a long[] array, allowing us to address 8192 * 2^32 * 8 bytes, which is - * approximately 35 terabytes of memory. + * maximum size of a long[] array, allowing us to address 8192 * (2^31 - 1) * 8 bytes, which is + * approximately 140 terabytes of memory. */ public class TaskMemoryManager { @@ -74,7 +74,7 @@ public class TaskMemoryManager { * Maximum supported data page size (in bytes). In principle, the maximum addressable page size is * (1L << OFFSET_BITS) bytes, which is 2+ petabytes. However, the on-heap allocator's * maximum page size is limited by the maximum amount of data that can be stored in a long[] - * array, which is (2^32 - 1) * 8 bytes (or 16 gigabytes). Therefore, we cap this at 16 gigabytes. + * array, which is (2^31 - 1) * 8 bytes (or about 17 gigabytes). Therefore, we cap this at 17 gigabytes. */ public static final long MAXIMUM_PAGE_SIZE_BYTES = ((1L << 31) - 1) * 8L; --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
