[
https://issues.apache.org/jira/browse/MAPREDUCE-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256607#comment-15256607
]
Ming Ma commented on MAPREDUCE-6315:
------------------------------------
Thanks [~jira.shegalov]! We definitely need to support this scenario. There was
some discussion about providing clean-up functionality in YARN-2261 and
MAPREDUCE-4428 so that history files can be moved to final location properly.
But it isn't clear when we plan to provide such functionality at YARN layer and
it seems like a larger effort. The patch here is more targeted and can take
care of the issue we have had, at least until the end-to-end clean-up
functionality is available. What do you think? Specific comments for the jira:
* This will enable "mapred job -logs" usage. How about the "jobhistory URL
http://...../jobhistory/job/job_ returns job not found" scenario, is it easy to
add the redirect at that level?
* globStatus might return empty list. So it might be better to change from {{if
(jhStats != null) }} to something like {{if (jhStats != null && jhStats.length
> 0)}}.
* Is the output format change in JobHistoryParser required? Wonder if there is
any backward compatibility issue if some tools have assumption about this.
> Implement retrieval of logs for crashed MR-AM via jhist in the staging
> directory
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6315
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6315
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client, mr-am
> Affects Versions: 2.7.0
> Reporter: Gera Shegalov
> Assignee: Gera Shegalov
> Priority: Critical
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-6315.001.patch, MAPREDUCE-6315.002.patch,
> MAPREDUCE-6315.003.patch
>
>
> When all AM attempts crash, there is no record of them in JHS. Thus no easy
> way to get the logs. This JIRA automates the procedure by utilizing the jhist
> file in the staging directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
