[
https://issues.apache.org/jira/browse/FLINK-5183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859800#comment-15859800
]
ASF GitHub Bot commented on FLINK-5183:
---------------------------------------
Github user GEOFBOT commented on the issue:
https://github.com/apache/flink/pull/3232
It may have worked with a smaller file, but there may be issues with
heavier jobs. When I ran a more computationally intensive and time consuming
job, the first job of the Python file ran successfully. The second job of the
file was then submitted:
```
<snip>
02/09/2017 16:39:43 DataSink (CsvSink)(4/5) switched to FINISHED
02/09/2017 16:39:43 Job execution switched to status FINISHED.
2017-02-09 16:40:26,470 INFO org.apache.flink.yarn.YarnClusterClient
- Waiting until all TaskManagers have connected
Waiting until all TaskManagers have connected
2017-02-09 16:40:26,476 INFO org.apache.flink.yarn.YarnClusterClient
- TaskManager status (5/5)
TaskManager status (5/5)
2017-02-09 16:40:26,476 INFO org.apache.flink.yarn.YarnClusterClient
- All TaskManagers are connected
All TaskManagers are connected
2017-02-09 16:40:26,480 INFO org.apache.flink.yarn.YarnClusterClient
- Submitting job with JobID: b226f5f18a78bc386bd1b1b6d30515ea.
Waiting for job completion.
Submitting job with JobID: b226f5f18a78bc386bd1b1b6d30515ea. Waiting for
job completion.
Connected to JobManager at
Actor[akka.tcp://flink@<snip>.ec2.internal:35598/user/jobmanager#68430682]
```
However, Flink does not receive or respond to this new job. Instead, the
client terminates with a timeout error:
```
Caused by:
org.apache.flink.runtime.client.JobClientActorSubmissionTimeoutException: Job
submission to the JobManager timed out. You may increase 'akka.client.timeout'
in case the JobManager needs more time to configure and confirm the job
submission.
at
org.apache.flink.runtime.client.JobSubmissionClientActor.handleCustomMessage(JobSubmissionClientActor.java:119)
at
org.apache.flink.runtime.client.JobClientActor.handleMessage(JobClientActor.java:239)
at
org.apache.flink.runtime.akka.FlinkUntypedActor.handleLeaderSessionID(FlinkUntypedActor.java:88)
at
org.apache.flink.runtime.akka.FlinkUntypedActor.onReceive(FlinkUntypedActor.java:68)
at
akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)
```
I tried setting `akka.client.timeout` to 20 minutes, but Flink is still not
receiving the second job. I suspect this may be an issue with this patch.
> [py] Support multiple jobs per Python plan file
> -----------------------------------------------
>
> Key: FLINK-5183
> URL: https://issues.apache.org/jira/browse/FLINK-5183
> Project: Flink
> Issue Type: Improvement
> Components: Python API
> Affects Versions: 1.1.3
> Reporter: Geoffrey Mon
> Priority: Minor
>
> Support running multiple jobs per Python plan file.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
