[
https://issues.apache.org/jira/browse/YARN-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Subru Krishnan updated YARN-2915:
---------------------------------
Release Note: A federation-based approach to transparently scale a single
YARN cluster to tens of thousands of nodes, by federating multiple YARN
standalone clusters.The proposed approach is to divide a large (10-100k nodes)
cluster into smaller units called sub-clusters, each with its own YARN RM and
compute nodes. The federation system will stitch these sub-clusters together
and make them appear as one large YARN cluster to the applications. The
applications running in this federated environment will see a single massive
YARN cluster and will be able to schedule tasks on any node of the federated
cluster. Under the hood, the federation system will negotiate with sub-clusters
resource managers and provide resources to the application. The goal is to
allow an individual job to “span” sub-clusters seamlessly. (was: A
federation-based approach to transparently scale a single YARN cluster to tens
of thousands of nodes, by federating multiple YARN standalone clusters)
> Enable YARN RM scale out via federation using multiple RM's
> -----------------------------------------------------------
>
> Key: YARN-2915
> URL: https://issues.apache.org/jira/browse/YARN-2915
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager, resourcemanager
> Reporter: Sriram Rao
> Assignee: Subru Krishnan
> Labels: federation
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: FEDERATION_CAPACITY_ALLOCATION_JIRA.pdf,
> Federation-BoF.pdf, YARN-Federation-Hadoop-Summit_final.pptx,
> Yarn_federation_design_v1.pdf, federation-prototype.patch
>
>
> This is an umbrella JIRA that proposes to scale out YARN to support large
> clusters comprising of tens of thousands of nodes. That is, rather than
> limiting a YARN managed cluster to about 4k in size, the proposal is to
> enable the YARN managed cluster to be elastically scalable.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
