[
https://issues.apache.org/jira/browse/HADOOP-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701550#action_12701550
]
Arun C Murthy commented on HADOOP-5632:
---------------------------------------
bq. If running tasks status updates can be done cheaply, we can piggyback these
updates on the lightweight heartbeats.
Uh, that pretty much all we do on the current heartbeat, along with
getTasksToKill, no? We should consider fixing getTasksToKill, after which we
are back at the current heartbeat.
bq. I was thinking of having the heavy weight heartbeats have a low frequency,
like every few minutes.
-1
That's too large, especially for jobs where latency is important. The 'heavy'
heartbeats cannot be much slower that what we have currently.
> Jobtracker leaves tasktrackers underutilized
> --------------------------------------------
>
> Key: HADOOP-5632
> URL: https://issues.apache.org/jira/browse/HADOOP-5632
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.18.0, 0.18.1, 0.18.2, 0.18.3, 0.19.0, 0.19.1, 0.20.0
> Environment: 2x HT 2.8GHz Intel Xeon, 3GB RAM, 4x 250GB HD linux
> boxes, 100 node cluster
> Reporter: Khaled Elmeleegy
> Attachments: hadoop-khaled-tasktracker.10s.uncompress.timeline.pdf,
> hadoop-khaled-tasktracker.150ms.uncompress.timeline.pdf, jobtracker.patch,
> jobtracker20.patch
>
>
> For some workloads, the jobtracker doesn't keep all the slots utilized even
> under heavy load.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
