[
https://issues.apache.org/jira/browse/HADOOP-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12542737
]
amareshwari edited comment on HADOOP-1900 at 11/15/07 3:20 AM:
----------------------------------------------------------------------------
Now considering only cluster size for varying heartbeat interval, the propasal
is as follows:
1. Heartbeat interval = max(2, clusterSize/50+1)
i.e. for every 50 nodes, we increase the heartbeat interval by 1 second.
2. Map completion events polling interval can take the same value as heartbeat
interval.
Apart from polling, the tasktracker will also fetch map events from the
JobTracker when a reducetask asks for events and it has nothing to give (this
is similar to the way tasktrackers ask for a new task whenever it finishes
executing a task)
Thoughts?
was (Author: amareshwari):
Now considering only cluster size for varying heartbeat interval, the
propasal is as follows:
1. Heartbeat interval = max(2, clusterSize/50+1)
i.e. for every 50 nodes, we increase the heartbeat interval by 1 second.
2. Map completion events polling interval can take the same value as heartbeat
interval.
mapEventsFetcherThread will be notified if a reduce task doesnt find map events
at the tasktracker.
Thoughts?
> the heartbeat and task event queries interval should be set dynamically by
> the JobTracker
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-1900
> URL: https://issues.apache.org/jira/browse/HADOOP-1900
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Attachments: patch-1900.txt, patch-1900.txt
>
>
> The JobTracker should scale the intervals that the TaskTrackers use to
> contact it dynamically, based on how the busy it is and the size of the
> cluster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
