Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/7741#discussion_r35773444
--- Diff:
unsafe/src/main/java/org/apache/spark/unsafe/memory/TaskMemoryManager.java ---
@@ -58,8 +58,8 @@
/** The number of entries in the page table. */
private static final int PAGE_TABLE_SIZE = 1 << PAGE_NUMBER_BITS;
- /** Maximum supported data page size */
- private static final long MAXIMUM_PAGE_SIZE = (1L << OFFSET_BITS);
+ /** Maximum supported data page size (in bytes). */
+ public static final long MAXIMUM_PAGE_SIZE_BYTES = (1L << OFFSET_BITS);
--- End diff --
I noticed that there's actually a bug in the existing code w.r.t on-heap
vs. off-heap mode: in on-heap mode, the maximum page size is bounded by both
the number of addressing bits _and_ the size of a long array, so this
theoretical upper limit only applies to off-heap mode.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
