[
https://issues.apache.org/jira/browse/HADOOP-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618922#action_12618922
]
Joydeep Sen Sarma commented on HADOOP-3136:
-------------------------------------------
i don't know how big a change we are envisaging here. my suggestion was
thinking if we wanted to minimize changes to current code base. I don't think
the protocol suggested would send any more heartbeats than we do in the worst
case today (or that at least it could be tuned to get that property) - but if
the plan going forward is to reduce heartbeats - then that's not good enough.
If we are planning non-trivial changes - it seems to me that it would be nice
as a first step to have scheduling decisions dispatched pro-actively to slaves
on new job arrival (after considering global resource availability). that
itself should save on latency of scheduling on an idling cluster. on a
backloaded cluster, scheduling reactively to heartbeats seems reasonable (in
fact - on a backloaded cluster - it's likely that each heartbeat would only
advertize one available slot. unless of course we move away from current
protocol for sending heartbeat on task completion).
the other significant improvement i can think of is maintaining a runnable task
queue on each TT (in addition to the ones running). that way there would be no
idle time lost to getting new tasks. the protocol of dispatching tasks to TT
runnable queues could happen in the background where one could make
optimizations to reduce network traffic. (for example - if the TT had a good
sized queue of runnable tasks then it could afford to delay sending heartbeats
on task completion (since delaying would no longer cause idle time)). This may
also allow the JT to make batched scheduling decisions (instead of immediately
looking for something to schedule when slots become free - the JT could wait
and accumulate some larger number of free slots before making a global
scheduling decision over all those slots and the available tasks. this might
make things more globally optimal while still not causing idle time (since
there every TT has a queue of runnable tasks)).
> Assign multiple tasks per TaskTracker heartbeat
> -----------------------------------------------
>
> Key: HADOOP-3136
> URL: https://issues.apache.org/jira/browse/HADOOP-3136
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Devaraj Das
> Assignee: Arun C Murthy
> Fix For: 0.19.0
>
>
> In today's logic of finding a new task, we assign only one task per heartbeat.
> We probably could give the tasktracker multiple tasks subject to the max
> number of free slots it has - for maps we could assign it data local tasks.
> We could probably run some logic to decide what to give it if we run out of
> data local tasks (e.g., tasks from overloaded racks, tasks that have least
> locality, etc.). In addition to maps, if it has reduce slots free, we could
> give it reduce task(s) as well. Again for reduces we could probably run some
> logic to give more tasks to nodes that are closer to nodes running most maps
> (assuming data generated is proportional to the number of maps). For e.g., if
> rack1 has 70% of the input splits, and we know that most maps are data/rack
> local, we try to schedule ~70% of the reducers there.
> Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
