[
https://issues.apache.org/jira/browse/FLINK-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184031#comment-16184031
]
ASF GitHub Bot commented on FLINK-7072:
---------------------------------------
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/4742
[FLINK-7072] [REST] Add Flip-6 client for submit/job/cancel
This PR builds on #4730 .
## What is the purpose of the change
This PR adds a new ClusterClient specifically for Flip-6 using the new REST
architecture. It supports submitting, canceling and stopping jobs.
The job submission is not done purely via REST. The jar upload is still
implemented through the blobserver. Thus, for submitting a job, we first query
the blobserver port, upload the jars, and then submit the jobgraph again via
REST.
The stopping and canceling of jobs uses the existing JobTerminationHandler
that was recently added.
## Brief change log
* define REST protocol for job submissions
* modify dispatcher to expose the blob server port
* add handlers to dispatcher endpoint for querying the blob server port and
submitting jobs
* add a new ClusterClient for Flip6 and integrate it into the CLI
## Verifying this change
This change added tests and can be verified as follows:
- start a flip6 cluster using `./bin/start-cluster.sh flip6`
- submit a job using `./bin/flink run -flip6 <jar>` (this will print the
job ID)
- stop/cancel the job using `./bin/flink [cancel|stop] -flip6 <jobid>`
- check logs that the job was successfully canceled
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not documented)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 7072
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4742.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4742
----
commit dab4a51434f4386bfa9c7c5bfc45a8875ede9302
Author: zentol <[email protected]>
Date: 2017-09-20T12:52:56Z
[hotfix] [REST] Extend empty request/parameters support
commit c5b1b734fe117b3a2c8645c7be17de41df4daeb0
Author: zentol <[email protected]>
Date: 2017-09-20T12:57:56Z
[hotfix] [REST] Fix error message if empty request does not conform to
RequestBody spec
commit 6a38409f87e0429dd2dccfeca93d79d9c9b9a858
Author: zentol <[email protected]>
Date: 2017-09-20T13:00:01Z
[hotfix] [REST] Add special handling for plain-text responses
commit d64cddd66308e125ed338214aa7c1e719f3548d4
Author: zentol <[email protected]>
Date: 2017-09-20T12:58:42Z
[hotfix] [REST] Add utility HandlerRequest constructor
commit c9427017f12ee67a7395fd669f968dfd3af2198c
Author: zentol <[email protected]>
Date: 2017-09-20T12:55:46Z
[FLINK-7072] [REST] Define protocol for job submit/cancel/stop
commit 02dbf5b644abb81cbb25531ad1cb012ef21a8420
Author: zentol <[email protected]>
Date: 2017-09-28T08:49:39Z
[FLINK-7072] [REST] Extend Dispatcher
commit 928171fbe56f42b297ce0ba1bb0566288a1d8383
Author: zentol <[email protected]>
Date: 2017-09-20T12:59:42Z
[FLINK-7072] [REST] Add handlers for job submit/cancel/stop
commit 44bdf5f778ebabcc9ca49fd833e184d21dc10634
Author: zentol <[email protected]>
Date: 2017-09-28T09:17:54Z
[FLINK-7072] [REST] CLI integration
----
> Create RESTful cluster endpoint
> -------------------------------
>
> Key: FLINK-7072
> URL: https://issues.apache.org/jira/browse/FLINK-7072
> Project: Flink
> Issue Type: Sub-task
> Components: Distributed Coordination
> Reporter: Till Rohrmann
> Assignee: Chesnay Schepler
> Labels: flip-6
> Fix For: 1.4.0
>
>
> In order to communicate with the cluster from the RESTful client, we have to
> implement a RESTful cluster endpoint. The endpoint shall support the
> following operations:
> * List jobs (GET): Get list of all running jobs on the cluster
> * Submit job (POST): Submit a job to the cluster (only supported in session
> mode)
> * Get job status (GET): Get the status of an executed job (and maybe the
> JobExecutionResult)
> * Lookup job leader (GET): Gets the JM leader for the given job
> This endpoint will run in session mode alongside the dispatcher/session
> runner and forward calls to this component which maintains a view on all
> currently executed jobs.
> In the per-job mode, the endpoint will return only the single running job and
> the address of the JobManager alongside which it is running. Furthermore, it
> won't accept job submissions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
