[
https://issues.apache.org/jira/browse/FLINK-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387465#comment-14387465
]
ASF GitHub Bot commented on FLINK-1771:
---------------------------------------
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/542#issuecomment-87840336
I know that the PR is touching many components. I'll try to split up my
work into smaller parts.
In this case I didn't expect in the beginning that I need to change so many
things.
The PR is mainly about adding support for executing a Flink job on YARN in
a "fire and forget" fashion.
Therefore, I needed to make some changes to the YARN client.
In the previous big change to YARN, I added support for a "detached" yarn
session. So that you can tell the Flink Yarn Client to start Flink on YARN
without connecting to the AM afterwards. Users have to manage such a yarn
session using other tools afterwards (for example `yarn application -kill` to
stop it)
This change brings this feature even further to support single flink jobs
being submitted to YARN. But since the Yarn client doesn't connect to the AM
once Flink has been started, there is no way to tell the AM to stop Flink on
YARN again.
In this change, I add a new Akka message for the ApplicationMaster `case
class StopAMAfterJob(jobId:JobID)`. The message tells the AM to monitor the JM
until the job has finished. Once that has happened, the AM stops Flink on YARN.
To get this `JobID` I needed to make some changes to the CliFrontend /
Client. The Client has two ways of submitting a Job to Flink: an attached mode
(default) and a detached mode.
The attached mode is returning the `JobExecutionResult`, the detached mode
was returning nothing. I created a new type called `JobSubmissionResult` which
is returned by the detached job submission. It only contains the job id. The
JobExecutionResult is extending the JobSubmissionResult.
> Add support for submitting single jobs to a detached YARN session
> -----------------------------------------------------------------
>
> Key: FLINK-1771
> URL: https://issues.apache.org/jira/browse/FLINK-1771
> Project: Flink
> Issue Type: Improvement
> Components: YARN Client
> Affects Versions: 0.9
> Reporter: Robert Metzger
> Assignee: Robert Metzger
>
> We need tests ensuring that the processing slots are set properly when
> starting Flink on YARN, in particular with the per job YARN session feature.
> Also, the YARN tests for detached YARN sessions / per job yarn clusters are
> polluting the local home-directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
