Repository: hadoop Updated Branches: refs/heads/branch-2 0882725c8 -> bda2a9c2a
YARN-7736. Fix itemization in YARN federation document Signed-off-by: Akira Ajisaka <[email protected]> (cherry picked from commit 245751ffdc4229715a0c031f57f20748ed16d8a6) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/bda2a9c2 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/bda2a9c2 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/bda2a9c2 Branch: refs/heads/branch-2 Commit: bda2a9c2a0ed77a000a31b9531b86cad1d0383ff Parents: 0882725 Author: Sen Zhao <[email protected]> Authored: Mon Mar 5 11:22:09 2018 -0800 Committer: Akira Ajisaka <[email protected]> Committed: Mon Mar 5 11:55:16 2018 -0800 ---------------------------------------------------------------------- .../hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md | 2 ++ 1 file changed, 2 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/bda2a9c2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md ---------------------------------------------------------------------- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md index 40d46cb..a1d3ab6 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md @@ -29,6 +29,7 @@ This architecture can provide very tight enforcement of scheduling invariants wi Federation is designed as a âlayerâ atop of existing YARN codebase, with limited changes in the core YARN mechanisms. Assumptions: + * We assume reasonably good connectivity across sub-clusters (e.g., we are not looking to federate across DC yet, though future investigations of this are not excluded). * We rely on HDFS federation (or equivalently scalable DFS solutions) to take care of scalability of the store side. @@ -71,6 +72,7 @@ to minimize overhead on the scheduling infrastructure (more in section on scalab  *Role of AMRMProxy* + 1. Protect the sub-cluster YARN RMs from misbehaving AMs. The AMRMProxy can prevent DDOS attacks by throttling/killing AMs that are asking too many resources. 2. Mask the multiple YARN RMs in the cluster, and can transparently allow the AM to span across sub-clusters. All container allocations are done by the YARN RM framework that consists of the AMRMProxy fronting the home and other sub-cluster RMs. 3. Intercepts all the requests, thus it can enforce application quotas, which would not be enforceable by sub-cluster RM (as each only see a fraction of the AM requests). --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
