GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/6203
[FLINK-9289][rest] Rework JobSubmitHandler to accept jar/artifact files
Builds on #6199.
## What is the purpose of the change
This PR reworks the `JobSubmitHandler` to accept jars/artifacts and the
jobgraph in a single REST request, and changes the `RestClusterClient` to make
use of this feature.
As a result the entire job submission now goes through the REST API.
There are still known issues in this PR but they won't impact the
correctness; the core implementation is complete, works and is covered by tests.
## Brief change log
* remove blobserver port headers/handlers and usages
* rework `JobSubmitRequestBody` to contain the names of the jobgraph, jar
and artifact files, providing semantics to the collection of uploaded files
* set visibility of utility methods in `ClientUtils` to `public`to allow
passing jar/artifact collections manually
* modified the `JobSubmitHandler` to
* read the jobgraph from an uploaded file
* upload uploaded jars/artifacts to the blob service and store keys in
the jobgraph
* modified the `RestClusterClient` to
* write jobgraph into a file for later upload
* make use of new `RestClient` API for multipart requests
* extended ´RestConstants` for jar/binary `content-types`, and made the
class an enum to get singleton properties
* extended documentation of `JobSubmitHeaders`
## Verifying this change
* this PR only modifies existing tests to accommodate changes
* the general functionality is covered by existing end-to-end tests
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (docs)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 9280_epsilon
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6203.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6203
----
commit 90d81f7bdde62f2b9bb1d1da071cc3d0b0d1b349
Author: zentol <chesnay@...>
Date: 2018-06-13T16:21:21Z
[FLINK-9624][runtime] Move jar/artifact upload out of jobgraph
commit a365697a031d16a778f25334c3bc2c7706d13693
Author: zentol <chesnay@...>
Date: 2018-06-11T09:45:12Z
[FLINK-9280][rest] Remove BlobServer port handler
commit 78b456eb861fd722a100ae6e153f1c0079649d8b
Author: zentol <chesnay@...>
Date: 2018-06-22T09:03:45Z
[FLINK-9289][rest] Rework JobSubmitHandler to accept jar/artifact files
----
---
