[
https://issues.apache.org/jira/browse/YARN-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720952#comment-17720952
]
ASF GitHub Bot commented on YARN-3660:
--------------------------------------
slfan1989 commented on PR #5625:
URL: https://github.com/apache/hadoop/pull/5625#issuecomment-1540257264
@goiri In the
[YARN-Federation](https://hadoop.apache.org/docs/r3.3.5/hadoop-yarn/hadoop-yarn-site/Federation.html)
documentation, it is mentioned that there is a `Global Policy Generator`
component and its code exists in the `YARN-7402` branch. Therefore, we need to
backport the code from the `YARN-7402` branch to the `trunk` branch.
I try to backport `YARN-3660` from the `YARN-7402` branch to the `trunk`
branch. During the backport process, I mostly kept the original code and fixed
issues flagged by Spotbugs.
I have compiled a list of Jira issues that need to be backported in the
future. I will submit `5-6` pull requests to backport this code to the trunk
branch.
The list is as follows. (These prs have already been merged into the
`YARN-7402` branch.)
- YARN-3360 [GPG] Federation Global Policy Generator (service hook only)
- YARN-6648 [GPG] Add SubClusterCleaner in Global Policy Generator
- YARN-7599 [GPG] ApplicationCleaner in Global Policy Generator
- YARN-7707 [GPG] Policy generator framework
- YARN-7708 [GPG] Load based policy generator
- YARN-8334 [GPG] Fix potential connection leak in GPGUtils
- YARN-8534 [GPG] Add max heap config option for Federation GPG
- YARN-8637 [GPG] Add FederationStateStore getAppInfo API for
GlobalPolicyGenerator
- YARN-8862 [GPG] Add Yarn Registry cleanup in ApplicationCleaner
- YARN-8934 [GPG] Add JvmMetricsInfo and pause monitor
- YARN-9013 [GPG] fix order of steps cleaning Registry entries in
ApplicationCleaner
Can you help to merge this pr into the trunk branch? Thank you very much! I
will continue to follow up the backport of other PRs.
The issue with the unit tests is not caused by our pr.
> [GPG] Federation Global Policy Generator (service hook only)
> ------------------------------------------------------------
>
> Key: YARN-3660
> URL: https://issues.apache.org/jira/browse/YARN-3660
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Carlo Curino
> Assignee: Botong Huang
> Priority: Major
> Labels: federation, gpg, pull-request-available
> Attachments: YARN-3660-YARN-7402.v1.patch,
> YARN-3660-YARN-7402.v2.patch, YARN-3660-YARN-7402.v3.patch,
> YARN-3660-YARN-7402.v3.patch, YARN-3660-YARN-7402.v3.patch,
> YARN-3660-YARN-7402.v4.patch
>
>
> In a federated environment, local impairments of one sub-cluster might
> unfairly affect users/queues that are mapped to that sub-cluster. A
> centralized component (GPG) runs out-of-band and edits the policies governing
> how users/queues are allocated to sub-clusters. This allows us to enforce
> global invariants (by dynamically updating locally-enforced invariants).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
