[
https://issues.apache.org/jira/browse/FLINK-1579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859955#comment-15859955
]
ASF GitHub Bot commented on FLINK-1579:
---------------------------------------
Github user uce commented on the issue:
https://github.com/apache/flink/pull/3286
Thanks for this PR. I think it's super cool (and long overdue) to finally
have this. I think the general approach is sound and has good things going for
it.
But I see two problems with maintaining the current implementation in the
long run:
- Changing a URL in the web dashboard requires that we also change it in
the history server file generation. It's easy to accidentally break the history
server and it's cumbersome to write tests when we extend it.
- We have all JSON generation in the JsonUtil. That results in a giant
class (as you say) that is really tightly coupled to the handlers but resides
in a different place.
I think we can word around this by moving the archival logic to the
handlers. I think it's fair that handlers that want to take part in historical
archiving implement their archival logic on their own. The handlers would need
to return their path and also their JSON. That way we keep all the handler
logic in one place for both the regular frontend and history server. A
"historic execution graph" might be a simple collection of `path, JSON` entries
that the history server persists on receive.
Another minor thing is that the final status of the operators is missing on
the history server. I don't know whether that is a front end issue or something
is missing in the JSON.
I think it's best to close this PR for now and think about how we want to
extend this a bit for better maintainability.
> Create a Flink History Server
> -----------------------------
>
> Key: FLINK-1579
> URL: https://issues.apache.org/jira/browse/FLINK-1579
> Project: Flink
> Issue Type: New Feature
> Components: Distributed Coordination
> Affects Versions: 0.9
> Reporter: Robert Metzger
> Assignee: Chesnay Schepler
>
> Right now its not possible to analyze the job results for jobs that ran on
> YARN, because we'll loose the information once the JobManager has stopped.
> Therefore, I propose to implement a "Flink History Server" which serves the
> results from these jobs.
> I haven't started thinking about the implementation, but I suspect it
> involves some JSON files stored in HDFS :)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
