Romain Yon created BEAM-2334:
--------------------------------
Summary: OutOfMemoryError in RandomAccessData.java:350
Key: BEAM-2334
URL: https://issues.apache.org/jira/browse/BEAM-2334
Project: Beam
Issue Type: Bug
Components: runner-core
Affects Versions: 2.0.0
Reporter: Romain Yon
Assignee: Kenneth Knowles
Got the following trace:
```
Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at java.util.Arrays.copyOf(Arrays.java:3236)
at
org.apache.beam.runners.dataflow.util.RandomAccessData.ensureCapacity(RandomAccessData.java:350)
at
org.apache.beam.runners.dataflow.util.RandomAccessData.access$300(RandomAccessData.java:51)
at
org.apache.beam.runners.dataflow.util.RandomAccessData$1.write(RandomAccessData.java:258)
at com.esotericsoftware.kryo.io.Output.flush(Output.java:185)
```
However the machine was not out of heap space. Seems like the problem is in
RandomAccessData.java:357:
Arrays.copyOf(buffer, Integer.MAX_VALUE)
Proof:
```
scala> Arrays.copyOf(v, Int.MaxValue)
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at java.util.Arrays.copyOf(Arrays.java:3236)
... 31 elided
scala> Arrays.copyOf(v, Int.MaxValue-2)
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236)
... 31 elided
scala> Arrays.copyOf(v, Int.MaxValue-1)
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at java.util.Arrays.copyOf(Arrays.java:3236)
... 31 elided
```
I'll submit a corresponding PR.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
