This is an automated email from the ASF dual-hosted git repository.
cloud-fan pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new d4cb4c917261 [MINOR][CORE] Fix the unit of the maximum page size in
PackedRecordPointer comment
d4cb4c917261 is described below
commit d4cb4c917261f15f81bdb32e2f375b33ba983e7b
Author: Yongbin Miao <[email protected]>
AuthorDate: Fri Oct 16 15:29:46 2026 -0700
[MINOR][CORE] Fix the unit of the maximum page size in PackedRecordPointer
comment
### What changes were proposed in this pull request?
This PR fixes a unit typo in the class-level comment of
`PackedRecordPointer`: the maximum addressable page size is `2^27 bytes = 128
megabytes`, not `2^27 bits`.
### Why are the changes needed?
The 27-bit field is a byte offset within a page, so the maximum addressable
page size is 2^27 **bytes** (128 MiB). This is consistent with:
- the same comment, which says the offsets "are not 8-byte-word-aligned",
and later that "Assuming word-alignment would allow for a 1 gigabyte maximum
page size" (2^27 * 8 bytes = 1 GiB);
- the constant right below it: `static final int MAXIMUM_PAGE_SIZE_BYTES =
1 << 27; // 128 megabytes`.
`2^27 bits` would only be 16 MiB, so the current wording is misleading.
### Does this PR introduce _any_ user-facing change?
No. Comment-only change.
### How was this patch tested?
Comment-only change; no tests are needed. The modified line is 99
characters, within the 100-character limit enforced by `dev/checkstyle.xml`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #59228 from DaemonforY/minor-fix-packed-record-pointer-comment.
Authored-by: Yongbin Miao <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 4c268bd8a14c604a552902b4e3afe174584a8c00)
Signed-off-by: Wenchen Fan <[email protected]>
---
.../main/java/org/apache/spark/shuffle/sort/PackedRecordPointer.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/core/src/main/java/org/apache/spark/shuffle/sort/PackedRecordPointer.java
b/core/src/main/java/org/apache/spark/shuffle/sort/PackedRecordPointer.java
index b36da80dbcff..4f34fe00032a 100644
--- a/core/src/main/java/org/apache/spark/shuffle/sort/PackedRecordPointer.java
+++ b/core/src/main/java/org/apache/spark/shuffle/sort/PackedRecordPointer.java
@@ -24,7 +24,7 @@ package org.apache.spark.shuffle.sort;
* <pre>
* [24 bit partition number][13 bit memory page number][27 bit offset in
page]
* </pre>
- * This implies that the maximum addressable page size is 2^27 bits = 128
megabytes, assuming that
+ * This implies that the maximum addressable page size is 2^27 bytes = 128
megabytes, assuming that
* our offsets in pages are not 8-byte-word-aligned. Since we have 2^13 pages
(based off the
* 13-bit page numbers assigned by {@link
org.apache.spark.memory.TaskMemoryManager}), this
* implies that we can address 2^13 * 128 megabytes = 1 terabyte of RAM per
task.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
