[
https://issues.apache.org/jira/browse/FLINK-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449491#comment-16449491
]
ASF GitHub Bot commented on FLINK-9194:
---------------------------------------
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/5902
[FLINK-9194][history] Add HistoryServer support to flip6
This PR subsumes #5888.
## What is the purpose of the change
This PR adds HistoryServer support to Flip6, i.e. allows the Dispatcher to
archive finished jobs. Additionally the HistoryServer was updated to reflect
the latest changes to the job overview page.
## Brief change log
* fix job overview merging of the HistoryServer (#5888)
* replace some manual JSON reading/writing with mappings to/from
`MultipleJobDetails`
* add a convenience `ArchivedJson` constructor that accepts a `ResponseBody`
* add `JsonArchivist` argument to `Dispatcher` constructor
* the archivist creates a `Collection<ArchivedJson>` which is passed to
`FsJobArchivist` for writing to disk
* `WebMonitorEndpoint` now implements `JsonArchivist`
* Modify relevant handlers to implement `JsonArchivist`
* this is different to the existing implementation of using static
factories
* archiving handlers are detected by the `WebMonitorEndpoint` which is
significantly less brittle as the existing approach of separately listing
archivers
* rework the `HistoryServerTest` to run multiple small jobs against a
`MiniClusterResource`
* this better reflects the actual behavior and ensures we don't break
things again
## Verifying this change
* run `HistoryServerTest`
* manually verify that Flip6 clusters can archive jobs, and ensure that
HistoryServer can read them.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 9194
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5902.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5902
----
commit aaf02376a0b4c16e454d0f3b733cee9edd20ef54
Author: hzyuqi1 <hzyuqi1@...>
Date: 2018-04-24T08:25:10Z
[FLINK-9246][HS] Adjust HistoryServer for job overview changes
commit 722590fa784f36051d8aab0ed01d3cae0c8f8200
Author: zentol <chesnay@...>
Date: 2018-04-24T08:26:32Z
[hotfix][history] Read/Write MultipleJobsDetails instead of manual JSON
commit 45bca6b3f996852ca85827903b5a9e90fa2793c1
Author: zentol <chesnay@...>
Date: 2018-04-18T12:33:04Z
[FLINK-9194][history] Add archiving routine to Dispatcher
commit de0684c548e5c9e966954082dd963358567997cc
Author: zentol <chesnay@...>
Date: 2018-04-18T12:33:16Z
[FLINK-9194][history] Add convenience ArchivedJson constructor
commit 8a0e9e28d429b2cb804dcef485adcd83c58e3b07
Author: zentol <chesnay@...>
Date: 2018-04-18T12:34:25Z
[FLINK-9194][history] Adjust handlers
commit 21d0bc0e5fc21d8e03a6c0b92014f0d49b62b0cb
Author: zentol <chesnay@...>
Date: 2018-04-23T13:13:41Z
[FLINK-9194][history] Rework and extend the HistoryServer test
----
> Finished jobs are not archived to HistoryServer
> -----------------------------------------------
>
> Key: FLINK-9194
> URL: https://issues.apache.org/jira/browse/FLINK-9194
> Project: Flink
> Issue Type: Bug
> Components: History Server, JobManager
> Affects Versions: 1.5.0
> Environment: Flink 2af481a
> Reporter: Gary Yao
> Assignee: Chesnay Schepler
> Priority: Blocker
> Labels: flip-6
> Fix For: 1.6.0
>
>
> In flip6 mode, jobs are not archived to the HistoryServer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
