[
https://issues.apache.org/jira/browse/HADOOP-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701220#action_12701220
]
Owen O'Malley commented on HADOOP-5632:
---------------------------------------
+1 to having separate rpcs. They should also have names that reflect their
function instead of "light" and "heavy" heartbeat.
Fundamentally, I think we should have a:
Task[] getNewTasks(String taskTrackerId, ResourceStatus resources, int
seqNumber) throws IOException;
And we should include the number of running maps and reduces there are in
ResourceStatus. That would allow us to leave the other heartbeat at the normal
interval to report tasks finishing and running task states as Devaraj said.
We would also need to have a sequence number that let the job tracker ignore
duplicate requests to avoid problems with the rpc responses getting lost,
similar to what we do with the heartbeats.
> Jobtracker leaves tasktrackers underutilized
> --------------------------------------------
>
> Key: HADOOP-5632
> URL: https://issues.apache.org/jira/browse/HADOOP-5632
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.18.0, 0.18.1, 0.18.2, 0.18.3, 0.19.0, 0.19.1, 0.20.0
> Environment: 2x HT 2.8GHz Intel Xeon, 3GB RAM, 4x 250GB HD linux
> boxes, 100 node cluster
> Reporter: Khaled Elmeleegy
> Attachments: hadoop-khaled-tasktracker.10s.uncompress.timeline.pdf,
> hadoop-khaled-tasktracker.150ms.uncompress.timeline.pdf, jobtracker.patch,
> jobtracker20.patch
>
>
> For some workloads, the jobtracker doesn't keep all the slots utilized even
> under heavy load.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
