[
https://issues.apache.org/jira/browse/YARN-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14215720#comment-14215720
]
Srikanth Kandula commented on YARN-2745:
----------------------------------------
Thanks Wangda.
Re: Yarn-314, that would be a good add! Our prototype implementation did not
use this functionality (same priority + locality but different resource size).
Because, we assumed that all of the tasks in a stage (e.g., map or reduce) have
the same resource demand. While this is a simplifying assumption and makes
things easier to estimate, it is not always correct especially when there is
skew. So the functionality in Yarn-314 would be a good thing to have and use
here too.
> YARN new pluggable scheduler which does multi-resource packing
> --------------------------------------------------------------
>
> Key: YARN-2745
> URL: https://issues.apache.org/jira/browse/YARN-2745
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager, scheduler
> Reporter: Robert Grandl
> Attachments: sigcomm_14_tetris_talk.pptx, tetris_paper.pdf
>
>
> In this umbrella JIRA we propose a new pluggable scheduler, which accounts
> for all resources used by a task (CPU, memory, disk, network) and it is able
> to achieve three competing objectives: fairness, improve cluster utilization
> and reduces average job completion time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
