[jira] [Commented] (FLINK-8119) Cannot submit jobs to YARN Session in FLIP-6 mode

2018-01-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324653#comment-16324653
 ] 

ASF GitHub Bot commented on FLINK-8119:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/5234


> Cannot submit jobs to YARN Session in FLIP-6 mode
> -
>
> Key: FLINK-8119
> URL: https://issues.apache.org/jira/browse/FLINK-8119
> Project: Flink
>  Issue Type: Sub-task
>  Components: Client
>Affects Versions: 1.5.0
>Reporter: Gary Yao
>Assignee: Till Rohrmann
>Priority: Blocker
>  Labels: flip-6
> Fix For: 1.5.0
>
>
> Cannot submit jobs to YARN Session in FLIP-6 mode because 
> {{FlinkYarnSessionCli}} becomes the _active_ CLI (should be 
> {{Flip6DefaultCLI}}).
> *Steps to reproduce*
> # Build Flink 1.5 {{101fef7397128b0aba23221481ab86f62b18118f}}
> # {{bin/yarn-session.sh -flip6 -d -n 1 -s 1 -jm 1024 -tm 1024}}
> # {{bin/flink run -flip6 ./examples/streaming/WordCount.jar}}
> # Verify that the job will not run.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8119) Cannot submit jobs to YARN Session in FLIP-6 mode

2018-01-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323812#comment-16323812
 ] 

ASF GitHub Bot commented on FLINK-8119:
---

Github user tillrohrmann commented on the issue:

https://github.com/apache/flink/pull/5234
  
Thanks for the review @GJL. Rebased onto the latest master. Merging once 
Travis gives green light.


> Cannot submit jobs to YARN Session in FLIP-6 mode
> -
>
> Key: FLINK-8119
> URL: https://issues.apache.org/jira/browse/FLINK-8119
> Project: Flink
>  Issue Type: Sub-task
>  Components: Client
>Affects Versions: 1.5.0
>Reporter: Gary Yao
>Assignee: Till Rohrmann
>Priority: Blocker
>  Labels: flip-6
> Fix For: 1.5.0
>
>
> Cannot submit jobs to YARN Session in FLIP-6 mode because 
> {{FlinkYarnSessionCli}} becomes the _active_ CLI (should be 
> {{Flip6DefaultCLI}}).
> *Steps to reproduce*
> # Build Flink 1.5 {{101fef7397128b0aba23221481ab86f62b18118f}}
> # {{bin/yarn-session.sh -flip6 -d -n 1 -s 1 -jm 1024 -tm 1024}}
> # {{bin/flink run -flip6 ./examples/streaming/WordCount.jar}}
> # Verify that the job will not run.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8119) Cannot submit jobs to YARN Session in FLIP-6 mode

2018-01-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308146#comment-16308146
 ] 

ASF GitHub Bot commented on FLINK-8119:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/5234

[FLINK-8119] [flip6] Wire correct Flip6 components in 
Flip6YarnClusterDescriptor

## What is the purpose of the change

Let the Flip6YarnClusterDescriptor create a RestClusterClient as 
ClusterClient.
Moreover, this commit makes the YarnResourceManager register under the REST 
port
at Yarn.

This PR is based on #5233.

## Brief change log

- Implement unsupported methods in `RestClusterClient`
- Register `YarnResourceManager` under REST port at Yarn
- Let `Flip6YarnClusterDescriptor` return a `RestClusterClient`

## Verifying this change

- Tested manually

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)
  - The S3 file system connector: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)
  - If yes, how is the feature documented? (not applicable)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink 
implementRestClusterClientMethods

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/5234.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5234


commit c48324702b0ebff3fd7a16e480ead90fc4c9a30b
Author: Till Rohrmann 
Date:   2017-12-07T12:57:24Z

[FLINK-8328] [flip6] Move Yarn ApplicationStatus polling out of 
YarnClusterClient

Introduce YarnApplicationStatusMonitor which does the Yarn 
ApplicationStatus polling in
the FlinkYarnSessionCli. This decouples the YarnClusterClient from the 
actual communication
with Yarn and, thus, gives a better separation of concerns.

commit 783bd1daf36b34a74cbe718a52a1043ba38d5a44
Author: Till Rohrmann 
Date:   2017-12-20T15:43:21Z

[FLINK-8329] [flip6] Move YarnClient to AbstractYarnClusterDescriptor

Moves the YarnClient from the YarnClusterClient to the 
AbstractYarnClusterDescriptor.
This makes the latter responsible for the lifecycle management of the 
client and gives
a better separation of concerns.

commit 192adb786a48d19b71e797355500652d51de6296
Author: Till Rohrmann 
Date:   2017-12-18T17:59:30Z

[FLINK-8332] [flip6] Move savepoint dispose into ClusterClient

Move the savepoint disposal logic from the CliFrontend into the 
ClusterClient. This gives
a better separation of concerns and allows the CliFrontend to be used with 
different
ClusterClient implementations.

commit a73b6e2850d4b2445a835914ba570d1057e59dfb
Author: Till Rohrmann 
Date:   2018-01-02T06:42:18Z

[FLINK-8333] [flip6] Separate deployment options from command options

This commit separates the parsing of command options and deployment options 
into two
steps. This makes it easier to make the CustomCommandLines non-static.

Moreover, this commit moves the CliFrontend into the cli sub package.

commit 57df8f9e1c5f4015cd6543ded43849a71003ef36
Author: Till Rohrmann 
Date:   2018-01-02T06:59:34Z

[FLINK-8338] [flip6] Make CustomCommandLines non static in CliFrontend

This commit changes how CustomCommandLines are registered at the 
CliFrontend.
Henceforth, the CliFrontend is initialized with the set of 
CustomCommandLines
instead of registering them statically. This improves maintainability and
testability.

commit 2ee8059b949b4a9c2e165f3e78716ae531e5c0e7
Author: Till Rohrmann 
Date:   2018-01-02T08:22:12Z

[FLINK-8339] [flip6] Let CustomCommandLine return ClusterDescriptor

Instead of directly retrieving or deploying a Flink cluster, the
CustomCommandLine now only returns a ClusterDescriptor which can be used
for these operations. This disentangles the ClusterDescriptor and the
CustomCommandLine a bit better supporting a proper lifecycle management
of the former.

commit 9ff62c7cfadc977d02f414ee26fae6b76a568eda
Author: Till Rohrmann 
Date:   2018-01-02T08:55:13Z

[FLINK-8340] [flip6] Remove passing of Configuration to CustomCommandLine

Since the