[ 
https://issues.apache.org/jira/browse/YARN-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Grandl updated YARN-2745:
--------------------------------
    Component/s: nodemanager
    Description: In this umbrella JIRA we propose an extension to existing 
scheduling techniques, which accounts for all resources used by a task (CPU, 
memory, disk, network) and it is able to achieve three competing objectives: 
fairness, improve cluster utilization and reduces average job completion time.  
(was: In this umbrella JIRA we propose a new pluggable scheduler, which 
accounts for all resources used by a task (CPU, memory, disk, network) and it 
is able to achieve three competing objectives: fairness, improve cluster 
utilization and reduces average job completion time.)
        Summary: Extend YARN to support multi-resource packing of tasks  (was: 
YARN new pluggable scheduler which does multi-resource packing)

Summary of main changes
* Update the container allocation logic in the RM scheduler. This change is the 
core. It enables “packing” tasks, preferring jobs with less remaining work and 
trades off fairness for efficiency. 

* Expand the AM->RM resource ask to expose tasks’ disk and network resource 
demands to the scheduler. 

*Support for cluster-wide resource tracking: want per-machine resource usage 
information available at the RM.  


> Extend YARN to support multi-resource packing of tasks
> ------------------------------------------------------
>
>                 Key: YARN-2745
>                 URL: https://issues.apache.org/jira/browse/YARN-2745
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager, resourcemanager, scheduler
>            Reporter: Robert Grandl
>         Attachments: sigcomm_14_tetris_talk.pptx, tetris_paper.pdf
>
>
> In this umbrella JIRA we propose an extension to existing scheduling 
> techniques, which accounts for all resources used by a task (CPU, memory, 
> disk, network) and it is able to achieve three competing objectives: 
> fairness, improve cluster utilization and reduces average job completion time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to