Yongjun Zhang commented on YARN-3021:

Hi [~jianhe],

Thanks for taking a further look. No worry about the delay, I guessed you were 

About your comment, the code 
 private void collectDelegationTokens(final String renewer,
                                       final Credentials credentials,
                                       final List<Token<?>> tokens)
                                           throws IOException {
    final String serviceName = getCanonicalServiceName();
    // Collect token of the this filesystem and then of its embedded children
    if (serviceName != null) { // fs has token, grab it
      final Text service = new Text(serviceName);
      Token<?> token = credentials.getToken(service); <============
      if (token == null) {
        token = getDelegationToken(renewer);
        if (token != null) {
          credentials.addToken(service, token);
The line highlighted with "<===" indicates that a token could be retrieved from 
the token map. In this case, are we sure that they always have a non-empty 
renewer? In addition, it's possible that we might change the 
{{skipTokenRenewer}} method in the future to do some additional checking.   
Seems safer to have this check. Do you think we should just keep this checking?


> YARN's delegation-token handling disallows certain trust setups to operate 
> properly over DistCp
> -----------------------------------------------------------------------------------------------
>                 Key: YARN-3021
>                 URL: https://issues.apache.org/jira/browse/YARN-3021
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 2.3.0
>            Reporter: Harsh J
>            Assignee: Yongjun Zhang
>         Attachments: YARN-3021.001.patch, YARN-3021.002.patch, 
> YARN-3021.003.patch, YARN-3021.004.patch, YARN-3021.005.patch, 
> YARN-3021.006.patch, YARN-3021.patch
> Consider this scenario of 3 realms: A, B and COMMON, where A trusts COMMON, 
> and B trusts COMMON (one way trusts both), and both A and B run HDFS + YARN 
> clusters.
> Now if one logs in with a COMMON credential, and runs a job on A's YARN that 
> needs to access B's HDFS (such as a DistCp), the operation fails in the RM, 
> as it attempts a renewDelegationToken(…) synchronously during application 
> submission (to validate the managed token before it adds it to a scheduler 
> for automatic renewal). The call obviously fails cause B realm will not trust 
> A's credentials (here, the RM's principal is the renewer).
> In the 1.x JobTracker the same call is present, but it is done asynchronously 
> and once the renewal attempt failed we simply ceased to schedule any further 
> attempts of renewals, rather than fail the job immediately.
> We should change the logic such that we attempt the renewal but go easy on 
> the failure and skip the scheduling alone, rather than bubble back an error 
> to the client, failing the app submission. This way the old behaviour is 
> retained.

This message was sent by Atlassian JIRA

Reply via email to