[
https://issues.apache.org/jira/browse/MAPREDUCE-961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Scott Chen updated MAPREDUCE-961:
---------------------------------
Attachment: ResourceScheduling.pdf
@Arun: Here is the overall design document. I hope this provides a coherent big
picture of the design. Let me know what you think.
Our goal is to collect resource information for tasks and tasktrackers and
schedule tasks based on the overall information. But since MAPREDUCE-220 (per
task information) is currently blocked by MAPREDUCE-901. We may first finish
MAPREDUCE-1218 (per node information) and do some resource based scheduling
based only on the per node resource information.
> ResourceAwareLoadManager to dynamically decide new tasks based on current
> CPU/memory load on TaskTracker(s)
> -----------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-961
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-961
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/fair-share
> Affects Versions: 0.22.0
> Reporter: dhruba borthakur
> Assignee: Scott Chen
> Fix For: 0.22.0
>
> Attachments: HIVE-961.patch, MAPREDUCE-961-v2.patch,
> MAPREDUCE-961-v3.patch, MAPREDUCE-961-v4.patch, ResourceScheduling.pdf
>
>
> Design and develop a ResouceAwareLoadManager for the FairShare scheduler that
> dynamically decides how many maps/reduces to run on a particular machine
> based on the CPU/Memory/diskIO/network usage in that machine. The amount of
> resources currently used on each task tracker is being fed into the
> ResourceAwareLoadManager in real-time via an entity that is external to
> Hadoop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
