[
https://issues.apache.org/jira/browse/MAPREDUCE-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759441#action_12759441
]
Arun C Murthy commented on MAPREDUCE-270:
-----------------------------------------
Nigel - This patch proved very very hard to test without mock-objects. For now,
I've attached a slightly arbitrary test-case which checks does the following:
# Simulates a very large cluster by setting a very high value of 30s for the
heartbeat-interval between the JobTracker and TaskTracker.
# Switches on the out-of-band heartbeat for the cluster.
# Submits a very small random-writer job with 2 maps and asserts that the job
completes within 120s.
The 120s deadline is carefully chosen with the idea that a randomwriter job
with 2 maps will need at least 4 heartbeats: setup-task, map_0, map_1 and
cleanup-task. However this is still arbitrary and not very scientific. So,
should we commit this test-case given that it is slightly flaky? Thoughts?
PS: The job completes in ~50s with out-of-band heartbeats turned on, and in
~3mins with it turned off. FYI
> TaskTracker could send an out-of-band heartbeat when the last running
> map/reduce completes
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-270
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-270
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.21.0
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-270.patch, MAPREDUCE-270.patch,
> MAPREDUCE-270.patch, MAPREDUCE-270_yhadoop20.patch,
> MAPREDUCE-270_yhadoop20.patch, MAPREDUCE-270_yhadoop20.patch
>
>
> Currently the TaskTracker strictly respects the heartbeat interval, this
> causes utilization issues when all running tasks complete. We could send an
> out-of-band heartbeat in that case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
