[
https://issues.apache.org/jira/browse/YARN-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13709262#comment-13709262
]
Jason Lowe commented on YARN-321:
---------------------------------
bq. Is there a reason to embed this inside the RM? I don't know if there were
reasons for the JHS to be separate, other than it being MR-specific.
IIRC the history server was embedded in the JT back in 1.x and was only split
out as a separate daemon to keep the RM from having a dependency on MR.
bq. That said, I agree it will be easier for the user if AHS starts along with
the RM. May be, that should be configurable and turned on by default?
That'd be my preference, and the proxyserver is already done this way. One can
run it either as part of the RM (default) or setup some configs and launch it
separately via {{yarn proxyserver}}.
> Generic application history service
> -----------------------------------
>
> Key: YARN-321
> URL: https://issues.apache.org/jira/browse/YARN-321
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Luke Lu
> Assignee: Vinod Kumar Vavilapalli
>
> The mapreduce job history server currently needs to be deployed as a trusted
> server in sync with the mapreduce runtime. Every new application would need a
> similar application history server. Having to deploy O(T*V) (where T is
> number of type of application, V is number of version of application) trusted
> servers is clearly not scalable.
> Job history storage handling itself is pretty generic: move the logs and
> history data into a particular directory for later serving. Job history data
> is already stored as json (or binary avro). I propose that we create only one
> trusted application history server, which can have a generic UI (display json
> as a tree of strings) as well. Specific application/version can deploy
> untrusted webapps (a la AMs) to query the application history server and
> interpret the json for its specific UI and/or analytics.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
