slfan1989 commented on code in PR #4954:
URL: https://github.com/apache/hadoop/pull/4954#discussion_r984135986
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/federation/FederationStateStoreService.java:
##########
@@ -378,4 +382,104 @@ public RouterMasterKeyResponse
getMasterKeyByDelegationKey(RouterMasterKeyReques
throws YarnException, IOException {
throw new NotImplementedException("Code is not implemented");
}
+
+ /**
+ * Create a thread that cleans up the app.
+ * @param stage rm-start/rm-stop.
+ */
+ public void createCleanUpFinishApplicationThread(String stage) {
+ String threadName = cleanUpThreadNamePrefix + "-" + stage;
+ Thread finishApplicationThread = new
Thread(createCleanUpFinishApplicationThread());
+ finishApplicationThread.setName(threadName);
+ finishApplicationThread.start();
+ }
+
+ /**
+ * Create a thread that cleans up the app.
+ *
+ * @return thread object.
+ */
+ private Runnable createCleanUpFinishApplicationThread() {
+ return () -> {
+
+ try {
+ // Get the current RM's App list based on subClusterId
+ GetApplicationsHomeSubClusterRequest request =
+ GetApplicationsHomeSubClusterRequest.newInstance(subClusterId);
+ GetApplicationsHomeSubClusterResponse response =
+ getApplicationsHomeSubCluster(request);
+ List<ApplicationHomeSubCluster> applications =
response.getAppsHomeSubClusters();
+
+ // Traverse the app list and clean up the app.
+ long successCleanUpAppCount = 0;
+ for (ApplicationHomeSubCluster application : applications) {
+ ApplicationId applicationId = application.getApplicationId();
+ if (!this.rmContext.getRMApps().containsKey(applicationId)) {
+ try {
+ DeleteApplicationHomeSubClusterResponse deleteResponse =
+ cleanUpFinishApplicationsWithRetries(applicationId);
+ if (deleteResponse != null) {
+ LOG.info("application = {} has been cleaned up successfully.",
applicationId);
+ successCleanUpAppCount++;
+ }
+ } catch (YarnException e) {
+ LOG.error("problem during application = {} cleanup.",
applicationId, e);
+ }
+ }
+ }
+
+ // print app cleanup log
+ LOG.info("cleanup finished applications size = {}, number = {}
successful cleanups.",
+ applications.size(), successCleanUpAppCount);
+ } catch (Exception e) {
+ LOG.error("problem during cleanup applications.", e);
+ }
+ };
+ }
+
+ /**
+ * Clean up the completed Application.
+ *
+ * @param applicationId app id.
+ * @return DeleteApplicationHomeSubClusterResponse.
+ * @throws Exception exception occurs.
+ */
+ public DeleteApplicationHomeSubClusterResponse
+ cleanUpFinishApplicationsWithRetries(ApplicationId applicationId) throws
Exception {
+ DeleteApplicationHomeSubClusterRequest request =
+ DeleteApplicationHomeSubClusterRequest.newInstance(applicationId);
+ return new
FederationStateStoreAction<DeleteApplicationHomeSubClusterResponse>() {
+ @Override
+ public DeleteApplicationHomeSubClusterResponse run() throws Exception {
+ return deleteApplicationHomeSubCluster(request);
+ }
+ }.runWithRetries();
+ }
+
+ /**
+ * Define an abstract class, abstract retry method,
+ * which can be used for other methods later.
+ *
+ * @param <T> abstract parameter
+ */
+ private abstract class FederationStateStoreAction<T> {
+ abstract T run() throws Exception;
+
+ T runWithRetries() throws Exception {
+ int retry = 0;
+ while (true) {
Review Comment:
There is no such method in the Yarn Federation module, I will make a
separate abstract class and put it in the yarn-server-common federation
package. This method can be used by other Yarn Federation modules.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
