Subru Krishnan updated YARN-2915:
    Attachment: federation-prototype.patch

Uploading a patch file that demonstrates the core YARN Federation end2end. We 
are able to federate multiple (8 in our case) YARN sub-clusters and run 
applications transparently across the federated cluster. We validated against 
MR, Spark and GridMix workloads with zero code change/recompiling.

We will be actively working in the next few weeks to clean up and break down 
this uber patch into smaller patches and uploading them to the correspoding 
sub-tasks by restructuring code, adding more test cases, comments etc

> Enable YARN RM scale out via federation using multiple RM's
> -----------------------------------------------------------
>                 Key: YARN-2915
>                 URL: https://issues.apache.org/jira/browse/YARN-2915
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, resourcemanager
>            Reporter: Sriram Rao
>            Assignee: Subru Krishnan
> Yarn_federation_design_v1.pdf, federation-prototype.patch
> This is an umbrella JIRA that proposes to scale out YARN to support large 
> clusters comprising of tens of thousands of nodes.   That is, rather than 
> limiting a YARN managed cluster to about 4k in size, the proposal is to 
> enable the YARN managed cluster to be elastically scalable.  

This message was sent by Atlassian JIRA

Reply via email to