[
https://issues.apache.org/jira/browse/HADOOP-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672356#action_12672356
]
Ravi Gummadi commented on HADOOP-4191:
--------------------------------------
Hi Amar,
FAILED_MAPS, FAILED_REDUCES are logged only from logFinished() --- i.e. for a
successful job.
Will be checking FINISHED_MAPS, FINISHED_REDUCES, FAILED_MAPS, FAILED_REDUCES
for the successful job for which all other job level keys are validated.
Is the following code being added to validateJobLevelKJeys() look fine ? (looks
like too much dependant on the fact that validateJobLevelKeys() is called only
after UtilsForTests.runJobSucceed() and that internally calls
UtilsForTests.runJob(). Hmm may be we can send expected number of
finished_maps, finished_reduces, failed_maps, failed_reduces as parameters ???)
if (status.equals("SUCCESS")) {
// Since UtilsForTests.runJob() sets number of maps to be run as 1 and
// number of reduces to be run as 0, we expect the following values
// for the case of runJobSucceed()
String fin_maps = values.get(Keys.FINISHED_MAPS);
assertTrue("Unexpected number of finished maps in history file",
fin_maps == 1);
String fin_reduces = values.get(Keys.FINISHED_REDUCES);
assertTrue("Unexpected number of finished reduces in history file",
fin_reduces == 0);
String failed_maps = values.get(Keys.FAILED_MAPS);
assertTrue("Unexpected number of failed maps in history file",
failed_maps == 0);
String failed_reduces = values.get(Keys.FAILED_REDUCES);
assertTrue("Unexpected number of finished reduces in history file",
failed_reduces == 0);
}
-Ravi
> Add a testcase for jobhistory
> -----------------------------
>
> Key: HADOOP-4191
> URL: https://issues.apache.org/jira/browse/HADOOP-4191
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred, test
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Attachments: HADOOP-4191.patch, HADOOP-4191.v2.patch
>
>
> Changes in job history might break the history parser which in turn might
> break some features like jobtracker-recovery, history-viewer etc. There
> should be a testcase that catches these incompatible changes early and
> informs about the expected change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
