Subru Krishnan commented on YARN-2915:

The proposed architecture leverages the notion of “federating” a number of 
standalone YARN clusters, each with its own RM and set of NMs, in a larger 
federated cluster. The federated cluster appears as a single giant cluster to 
the user with a single public end-point (aka router) that confirms to the YARN 
application client protocol. The individual RMs will participate as members of 
the federated cluster by heart beating their capabilities, enabling dynamic 
self-configuration of the federated cluster. The applications running in this 
environment will have access to the entire federated YARN cluster and will be 
able to schedule tasks on any node of the federated cluster. We plan to extend 
the distributed scheduling proposed in YARN-2877, specifically YARN-2884 to 
allow AMs to request containers across the federated cluster. 

> Enable YARN RM scale out via federation using multiple RM's
> -----------------------------------------------------------
>                 Key: YARN-2915
>                 URL: https://issues.apache.org/jira/browse/YARN-2915
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, resourcemanager
>            Reporter: Sriram Rao
>            Assignee: Subru Krishnan
> This is an umbrella JIRA that proposes to scale out YARN to support large 
> clusters comprising of tens of thousands of nodes.   That is, rather than 
> limiting a YARN managed cluster to about 4k in size, the proposal is to 
> enable the YARN managed cluster to be elastically scalable.  

This message was sent by Atlassian JIRA

Reply via email to