Repository: hadoop Updated Branches: refs/heads/branch-3.0 d28db463c -> 4df200bc6
YARN-7736. Fix itemization in YARN federation document Signed-off-by: Akira Ajisaka <[email protected]> (cherry picked from commit 245751ffdc4229715a0c031f57f20748ed16d8a6) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/4df200bc Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/4df200bc Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/4df200bc Branch: refs/heads/branch-3.0 Commit: 4df200bc6df47161a4f8bfe41b5c1752ac9503b2 Parents: d28db46 Author: Sen Zhao <[email protected]> Authored: Mon Mar 5 11:22:09 2018 -0800 Committer: Akira Ajisaka <[email protected]> Committed: Mon Mar 5 11:55:01 2018 -0800 ---------------------------------------------------------------------- .../hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md | 2 ++ 1 file changed, 2 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/4df200bc/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md ---------------------------------------------------------------------- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md index ef0f713..f83d57e 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md @@ -29,6 +29,7 @@ This architecture can provide very tight enforcement of scheduling invariants wi Federation is designed as a âlayerâ atop of existing YARN codebase, with limited changes in the core YARN mechanisms. Assumptions: + * We assume reasonably good connectivity across sub-clusters (e.g., we are not looking to federate across DC yet, though future investigations of this are not excluded). * We rely on HDFS federation (or equivalently scalable DFS solutions) to take care of scalability of the store side. @@ -71,6 +72,7 @@ to minimize overhead on the scheduling infrastructure (more in section on scalab  *Role of AMRMProxy* + 1. Protect the sub-cluster YARN RMs from misbehaving AMs. The AMRMProxy can prevent DDOS attacks by throttling/killing AMs that are asking too many resources. 2. Mask the multiple YARN RMs in the cluster, and can transparently allow the AM to span across sub-clusters. All container allocations are done by the YARN RM framework that consists of the AMRMProxy fronting the home and other sub-cluster RMs. 3. Intercepts all the requests, thus it can enforce application quotas, which would not be enforceable by sub-cluster RM (as each only see a fraction of the AM requests). --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
