[
https://issues.apache.org/jira/browse/BEAM-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Cwik reassigned BEAM-2334:
-------------------------------
Assignee: Romain Yon (was: Kenneth Knowles)
> OutOfMemoryError in RandomAccessData.java:350
> ---------------------------------------------
>
> Key: BEAM-2334
> URL: https://issues.apache.org/jira/browse/BEAM-2334
> Project: Beam
> Issue Type: Bug
> Components: runner-core
> Affects Versions: 2.0.0
> Reporter: Romain Yon
> Assignee: Romain Yon
>
> Got the following trace:
> ```
> Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
> at java.util.Arrays.copyOf(Arrays.java:3236)
> at
> org.apache.beam.runners.dataflow.util.RandomAccessData.ensureCapacity(RandomAccessData.java:350)
> at
> org.apache.beam.runners.dataflow.util.RandomAccessData.access$300(RandomAccessData.java:51)
> at
> org.apache.beam.runners.dataflow.util.RandomAccessData$1.write(RandomAccessData.java:258)
> at com.esotericsoftware.kryo.io.Output.flush(Output.java:185)
> ```
> However the machine was not out of heap space. Seems like the problem is in
> RandomAccessData.java:357:
> Arrays.copyOf(buffer, Integer.MAX_VALUE)
> Proof:
> ```
> scala> Arrays.copyOf(v, Int.MaxValue)
> java.lang.OutOfMemoryError: Requested array size exceeds VM limit
> at java.util.Arrays.copyOf(Arrays.java:3236)
> ... 31 elided
> scala> Arrays.copyOf(v, Int.MaxValue-2)
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:3236)
> ... 31 elided
> scala> Arrays.copyOf(v, Int.MaxValue-1)
> java.lang.OutOfMemoryError: Requested array size exceeds VM limit
> at java.util.Arrays.copyOf(Arrays.java:3236)
> ... 31 elided
> ```
> I'll submit a corresponding PR.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
