Murtadha Hubail has submitted this change and it was merged. Change subject: [NO ISSUE][RT] Remove Hard Memory Limit in ByteArrayAccessibleOutputStream ......................................................................
[NO ISSUE][RT] Remove Hard Memory Limit in ByteArrayAccessibleOutputStream - user model changes: no - storage format changes: no - interface changes: no Details: - Remove the hardcoded 64MB memory limit from ByteArrayAccessibleOutputStream as the limit shouldn't be controlled by this structure but rather by operators using it. Change-Id: Ia88861c44802e64dbfceb5e8efc75d28bd54b501 Reviewed-on: https://asterix-gerrit.ics.uci.edu/2073 Tested-by: Jenkins <[email protected]> Contrib: Jenkins <[email protected]> Integration-Tests: Jenkins <[email protected]> Reviewed-by: Till Westmann <[email protected]> --- M hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/ByteArrayAccessibleOutputStream.java 1 file changed, 12 insertions(+), 11 deletions(-) Approvals: Anon. E. Moose #1000171: Till Westmann: Looks good to me, approved Jenkins: Verified; ; Verified Objections: Jenkins: Violations found diff --git a/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/ByteArrayAccessibleOutputStream.java b/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/ByteArrayAccessibleOutputStream.java index bf0e1dd..1a806cd 100644 --- a/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/ByteArrayAccessibleOutputStream.java +++ b/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/ByteArrayAccessibleOutputStream.java @@ -23,8 +23,7 @@ public class ByteArrayAccessibleOutputStream extends ByteArrayOutputStream { - private static final int MAX_SIZE = 1024 * 1024 * 32; - private static final double BUFFER_INCREMENT_FACTOR = 1.5; + private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE; public ByteArrayAccessibleOutputStream() { super(); @@ -97,22 +96,24 @@ private void grow(int minCapacity) { // overflow-conscious code int oldCapacity = buf.length; - if (oldCapacity == MAX_SIZE) { - throw new IllegalArgumentException("Buffer is too large..."); - } - int newCapacity = Math.min((int) (oldCapacity * BUFFER_INCREMENT_FACTOR), MAX_SIZE); + // increase by a factor of 1.5 + int newCapacity = oldCapacity + (oldCapacity >> 1); if (newCapacity - minCapacity < 0) { newCapacity = minCapacity; } - if (newCapacity < 0) { - if (minCapacity < 0) { - throw new OutOfMemoryError(); - } - newCapacity = Integer.MAX_VALUE; + if (newCapacity - MAX_ARRAY_SIZE > 0) { + newCapacity = hugeCapacity(minCapacity); } buf = Arrays.copyOf(buf, newCapacity); } + private static int hugeCapacity(int minCapacity) { + if (minCapacity < 0) { // overflow + throw new RuntimeException("Memory allocation limit (" + MAX_ARRAY_SIZE + " bytes) exceeded"); + } + return (minCapacity > MAX_ARRAY_SIZE) ? Integer.MAX_VALUE : MAX_ARRAY_SIZE; + } + /** * Return the current length of this stream (not capacity). */ -- To view, visit https://asterix-gerrit.ics.uci.edu/2073 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ia88861c44802e64dbfceb5e8efc75d28bd54b501 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]>
