[
https://issues.apache.org/jira/browse/HADOOP-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628611#action_12628611
]
Vinod K V commented on HADOOP-2165:
-----------------------------------
As Amareshwari pointed out, users might hit dead hyper-links in cases when a TT
is down, or when TaskLogServlet of a TT is unresponsive. Will change the patch
to avoid dead hyper-links, by doing a HTTP HEAD request to a tasklog servlet
and checking if returns alive.
There is also a special case when userlogs are expired from the TT. Default
userlog-expiry-interval is 24 hrs, and default expiry interval for job-history
files on JT is 30 days. So, there will be a window when we show userlogs links
on taskdetailshistory.jsp that will give no useful information. This is similar
to the situation with taskdetails.jsp when jobs are still in JT memory, but
userlogs are expired. Both can be solved by saving the userlogs in DFS beyond
the userlog-expiry-interval. But that is better solved as part of a bigger
hadoop log-aggregation effort. Once we have such a feature, we can either point
these hyper-links to or pull the logs from there.
To conclude, adding (non-dead) hyper-links is the best we wish to do here.
> Augment JobHistory to store tasks' userlogs
> -------------------------------------------
>
> Key: HADOOP-2165
> URL: https://issues.apache.org/jira/browse/HADOOP-2165
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Arun C Murthy
> Assignee: Vinod K V
> Attachments: patch_userlog_1.4.3.txt
>
>
> It will be very useful to be able to see the job's userlogs (the
> stdout/stderr/syslog of the tasks) from the JobHistory page. It will greatly
> aid in debugging etc.
> At the very minimum we should have links from the JobHistory to the logs on
> the TT.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
