[
https://issues.apache.org/jira/browse/YARN-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221174#comment-14221174
]
Chen He commented on YARN-2877:
-------------------------------
This is a interesting idea. Distributed scheduling and global scheduling have
their own pros and cons. For short, global scheduling can achieve optimal
matching between tasks and resources but may have scalability problem when
system becomes larger and larger. Distributed scheduling is scalable but may
reach sub-optimal if there is no communication between those distributed
schedulers.
The LocalRM can reduce the RM's burden by doing communications to local AMs. It
is a good idea. IMHO, the worker nodes become increasingly powerful and large
(more mems and cores). Is that possible that the LocalRM affects NM's
performance if there are many AMs running on a single server?
> Extend YARN to support distributed scheduling
> ---------------------------------------------
>
> Key: YARN-2877
> URL: https://issues.apache.org/jira/browse/YARN-2877
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager, resourcemanager
> Reporter: Sriram Rao
>
> This is an umbrella JIRA that proposes to extend YARN to support distributed
> scheduling. Briefly, some of the motivations for distributed scheduling are
> the following:
> 1. Improve cluster utilization by opportunistically executing tasks otherwise
> idle resources on individual machines.
> 2. Reduce allocation latency. Tasks where the scheduling time dominates
> (i.e., task execution time is much less compared to the time required for
> obtaining a container from the RM).
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
