Jian He commented on YARN-2010:

bq. Any subsequent attempts to transition the RM to active fail because 
RMActiveServices is not INITED, as in the Standby case
 I think YARN-2588 fixed this.  are you running into this error with the patch ?
-  How about moving “addApplicationSync” into RMAppRecoveredTransition. We can 
catch the exception inside the transition and return failed state directly ?
      // If security is enabled and the application is NOT in a final state,
      // parse the credentials and renew delegation token
      if (UserGroupInformation.isSecurityEnabled() &&
          !isApplicationInFinalState(appState.getState())) {
        Credentials credentials = parseCredentials(appContext);
        // synchronously renew delegation token on recovery.
            credentials, appContext.getCancelTokensWhenComplete());

      // Actual recovery of the application
      application.handle(new RMAppEvent(appId, RMAppEventType.RECOVER));
    } catch (Exception e) {
      LOG.error("Failed to recover application + " + appId, e);
      // Fail the application if it is a running application.
      if (!isApplicationInFinalState(appState.getState())) {
            new RMAppRejectedEvent(appId, e.getMessage()));
      throw e;
- changes in TestWorkPreservingRMRestart
It was purposely done to force RM to fail if the queue is missing for the app 
and indicate admin to config the queue properly.

> If RM fails to recover an app, it can never transition to active again
> ----------------------------------------------------------------------
>                 Key: YARN-2010
>                 URL: https://issues.apache.org/jira/browse/YARN-2010
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.3.0
>            Reporter: bc Wong
>            Assignee: Karthik Kambatla
>            Priority: Blocker
>         Attachments: YARN-2010.1.patch, YARN-2010.patch, 
> issue-stacktrace.rtf, yarn-2010-2.patch, yarn-2010-3.patch, 
> yarn-2010-3.patch, yarn-2010-4.patch, yarn-2010-5.patch, yarn-2010-6.patch
> Sometimes, the RM fails to recover an application. It could be because of 
> turning security on, token expiry, or issues connecting to HDFS etc. The 
> causes could be classified into (1) transient, (2) specific to one 
> application, and (3) permanent and apply to multiple (all) applications. 
> Today, the RM fails to transition to Active and ends up in STOPPED state and 
> can never be transitioned to Active again.
> The initial stacktrace reported is at 
> https://issues.apache.org/jira/secure/attachment/12676476/issue-stacktrace.rtf

This message was sent by Atlassian JIRA

Reply via email to