Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19266#discussion_r139656129
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java ---
@@ -39,7 +39,7 @@
private final long length;
public LongArray(MemoryBlock memory) {
- assert memory.size() < (long) Integer.MAX_VALUE * 8: "Array size > 4
billion elements";
+ assert memory.size() < (long) Integer.MAX_VALUE * 8: "Array size > 2.1
billion elements";
--- End diff --
I wasn't sure whether the JVM array size limit applied here, because this
represents a non-native array. Still wanted to fix the comment as I saw it. If
an array exists, its length is valid, so didn't think that part of MemoryBlock
represented an issue.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
