[
https://issues.apache.org/jira/browse/YARN-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776308#comment-17776308
]
ASF GitHub Bot commented on YARN-9013:
--------------------------------------
goiri commented on code in PR #6147:
URL: https://github.com/apache/hadoop/pull/6147#discussion_r1362534634
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java:
##########
@@ -4367,7 +4367,7 @@ public static boolean isAclEnabled(Configuration conf) {
* the default value is 0s.
*/
public static final long
DEFAULT_ROUTER_USER_CLIENT_THREAD_POOL_KEEP_ALIVE_TIME =
- TimeUnit.SECONDS.toMillis(0); // 0s
+ TimeUnit.SECONDS.toMillis(30); // 0s
Review Comment:
The comment needs to be updated.
Also not sure what's the agreement on changing default values.
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator/src/test/java/org/apache/hadoop/yarn/server/globalpolicygenerator/applicationcleaner/TestDefaultApplicationCleaner.java:
##########
@@ -159,7 +163,39 @@ public class TestableDefaultApplicationCleaner
extends DefaultApplicationCleaner {
@Override
public Set<ApplicationId> getAppsFromRouter() throws YarnRuntimeException {
+ if (appIdToAddConcurrently != null) {
+ SubClusterId scId = SubClusterId.newInstance("MySubClusterId");
+ try {
+ ApplicationHomeSubCluster appHomeSubCluster =
+ ApplicationHomeSubCluster.newInstance(appIdToAddConcurrently,
scId);
+ AddApplicationHomeSubClusterRequest request =
+
AddApplicationHomeSubClusterRequest.newInstance(appHomeSubCluster);
+ stateStore.addApplicationHomeSubCluster(request);
+ } catch (YarnException e) {
+ throw new YarnRuntimeException(e);
+ }
+ registryClient.writeAMRMTokenForUAM(appIdToAddConcurrently,
scId.toString(),
+ new Token<>());
+ }
return routerAppIds;
}
}
+
+ @Test
+ public void testConcurrentNewApp() throws YarnException {
+ appIdToAddConcurrently = ApplicationId.newInstance(1, 1);
+
+ appCleaner.run();
+
+ // The concurrently added app should be still there
+ GetApplicationsHomeSubClusterRequest appHomeSubClusterRequest =
+ GetApplicationsHomeSubClusterRequest.newInstance();
+ GetApplicationsHomeSubClusterResponse applicationsHomeSubCluster =
+ stateStore.getApplicationsHomeSubCluster(appHomeSubClusterRequest);
+ int size = applicationsHomeSubCluster.getAppsHomeSubClusters().size();
+ Assert.assertEquals(1, size);
Review Comment:
For consistency, either extract both or none.
> [GPG] fix order of steps cleaning Registry entries in ApplicationCleaner
> ------------------------------------------------------------------------
>
> Key: YARN-9013
> URL: https://issues.apache.org/jira/browse/YARN-9013
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Botong Huang
> Assignee: Botong Huang
> Priority: Major
> Labels: pull-request-available
> Attachments: YARN-9013-YARN-7402.v1.patch,
> YARN-9013-YARN-7402.v2.patch
>
>
> ApplicationCleaner today deletes the entry for all finished (non-running)
> application in YarnRegistry using this logic:
> # GPG gets the list of running applications from Router.
> # GPG gets the full list of applications in registry
> # GPG deletes in registry every app in 2 that’s not in 1
> The problem is that jobs that started between 1 and 2 meets the criteria in
> 3, and thus get deleted by mistake. The fix/right order should be 2->1->3,
> rather than 1->2->3.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
