[
https://issues.apache.org/jira/browse/FLINK-934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031071#comment-14031071
]
ASF GitHub Bot commented on FLINK-934:
--------------------------------------
GitHub user rmetzger opened a pull request:
https://github.com/apache/incubator-flink/pull/18
Do not limit the client's heap space (See FLINK-934)
Similar to FLINK-934 and https://github.com/apache/incubator-flink/pull/14
we should not limit the heap space of the job submission client to 512MB.
A user actually had the following exception while using Flink:
```
java.lang.OutOfMemoryError: Java heap space
at
eu.stratosphere.runtime.io.serialization.DataOutputSerializer.resize(DataOutputSerializer.java:243)
at
eu.stratosphere.runtime.io.serialization.DataOutputSerializer.write(DataOutputSerializer.java:87)
at
eu.stratosphere.nephele.jobgraph.JobGraph.writeRequiredJarFiles(JobGraph.java:706)
at eu.stratosphere.nephele.jobgraph.JobGraph.write(JobGraph.java:641)
at eu.stratosphere.nephele.ipc.RPC$Invocation.write(RPC.java:153)
at
eu.stratosphere.nephele.ipc.Client$Connection.sendParam(Client.java:469)
at eu.stratosphere.nephele.ipc.Client.call(Client.java:689)
at eu.stratosphere.nephele.ipc.RPC$Invoker.invoke(RPC.java:250)
at com.sun.proxy.$Proxy1.submitJob(Unknown Source)
at
eu.stratosphere.nephele.client.JobClient.submitJobAndWait(JobClient.java:258)
at eu.stratosphere.client.program.Client.run(Client.java:284)
at eu.stratosphere.client.program.Client.run(Client.java:268)
at eu.stratosphere.client.program.Client.run(Client.java:262)
at
eu.stratosphere.client.program.ContextEnvironment.execute(ContextEnvironment.java:50)
at
eu.stratosphere.example.java.wordcount.WordCount.main(WordCount.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
eu.stratosphere.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:384)
at
eu.stratosphere.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:302)
at eu.stratosphere.client.program.Client.run(Client.java:220)
at
eu.stratosphere.client.CliFrontend.executeProgram(CliFrontend.java:327)
at eu.stratosphere.client.CliFrontend.run(CliFrontend.java:314)
at
eu.stratosphere.client.CliFrontend.parseParameters(CliFrontend.java:927)
at eu.stratosphere.client.CliFrontend.main(CliFrontend.java:951)
```
I verified that the change resolves the problem.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rmetzger/incubator-flink
stratosphere_client_fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/18.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18
----
commit 8e1bff41a21d75f5e7dc78fbc5c30835a3802781
Author: Robert Metzger <[email protected]>
Date: 2014-06-13T19:40:34Z
Do not limit the client's heap space (See FLINK-934) [ci skip]
----
> Job and TaskManager startup scripts always set JVM heap size
> ------------------------------------------------------------
>
> Key: FLINK-934
> URL: https://issues.apache.org/jira/browse/FLINK-934
> Project: Flink
> Issue Type: Improvement
> Reporter: Ufuk Celebi
> Assignee: Ufuk Celebi
> Fix For: 0.6-incubating
>
>
> The default distribution config {{stratosphere-conf.yaml}} contains keys
> {{jobmanager.heap.mb}} and {{taskmanager.heap.mb}} for the heap sizes of the
> JobManager and TaskManager JVMs. If they are not set, {{config.sh}} has
> default values for them.
> We _always_ use one of these values and set the {{Xms}} and {{Xmx}}
> properties. After a short discussion with [~rmetzger] and [~StephanEwen] we
> actually expected that they should not be set if not found in
> {{stratosphere-conf.yaml}}. This would result in a fall back to the
> respective JVM default (imho 1/4th of the physical memory since Java 6).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
