[
https://issues.apache.org/jira/browse/HADOOP-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535840
]
Amareshwari Sri Ramadasu commented on HADOOP-1900:
--------------------------------------------------
bq. I wonder if instead we should just make it clusterSize/50+1? That way,
small clusters will get a heartbeat of just one second, which should make them
more responsive.
Agreed.
bq. Now, MapEventsFetcherThread polls jobtracker for completed map tasks for
every 5 secs (MIN_POLL_INTERVAL). Shall we change polling interval also in the
similar fashion as heartbeat interval? But, here some reduce tasks could be
idle for longer time.
We can have MapEventsFetcherThread polling jobtracker for completed map tasks
in the similar fashion as heartbeat interval with MIN_POLL_INTERVAL=5secs and
MIN_POLL_INTERVAL_MAX=30secs. And Whenever tasktracker finds there are no
mapevents and reduce task is waiting for the map events, it will wakeup the
thread to fetch map events from job tracker.
> the heartbeat and task event queries interval should be set dynamically by
> the JobTracker
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-1900
> URL: https://issues.apache.org/jira/browse/HADOOP-1900
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
>
> The JobTracker should scale the intervals that the TaskTrackers use to
> contact it dynamically, based on how the busy it is and the size of the
> cluster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
