[
https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755979#action_12755979
]
Iyappan Srinivasan commented on MAPREDUCE-157:
----------------------------------------------
+1 for QA
1) Tried multiple restarts. It resubmits properly and links work correct.
2) Looked at the logs of TT to see if the numbers showed at web page are really
correct.
3) Checked all the links including the "Analyse" logs page to see if there is a
mistake.
4) Got more than 100 jobs in history page to see if pagination works
5) searched based on user and based on job to see if it works.
6) Tried deleting some job.xml and job files in done folder to see how
jobhistory parser reacts.
7) Tried corrupting some job fles and then restarting JT. They are blocked from
viewing.
8) Tried killing some jobs, killing some attempt ids, using kill -9 to see if
it is getting handled properly.
9) Tried killing some jobs, killing some attempt ids, using bin/hadoop job
-kill to see if it is getting handled properly.
10) When mapreduce.cluster.jobhistory.maxage -> set to 10*60*60 (10 mins),then
the job files are getting cleaned up after 10 minutes.
11) Run some jobs and do a JT restart. After restart, visit those jobs back in
jobs history and see if they exist.
12)Set hadoop.job.history.location to a unreachable location and see if the JT
does not come up, throwing a proper
error.
> Job History log file format is not friendly for external tools.
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-157
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-157
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Affects Versions: 0.20.1
> Reporter: Owen O'Malley
> Assignee: Jothi Padmanabhan
> Fix For: 0.21.0
>
> Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch,
> mapred-157-15Sep.patch, mapred-157-16Sep.patch, mapred-157-4Sep.patch,
> mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch,
> MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch
>
>
> Currently, parsing the job history logs with external tools is very difficult
> because of the format. The most critical problem is that newlines aren't
> escaped in the strings. That makes using tools like grep, sed, and awk very
> tricky.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
