[ https://issues.apache.org/jira/browse/YARN-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14370440#comment-14370440 ]
Jian He commented on YARN-3021: ------------------------------- thanks Yongjun, some comments on the patch ! - DelegationTokenRenewer: the skipTokenRenewal check should be done under the existing code {{if (token.getKind().equals(new Text("HDFS_DELEGATION_TOKEN")))}} as below. And I think only doing this check is enough, we don't need checks in other places. {code} if (token.isManaged()) { if (token.getKind().equals(new Text("HDFS_DELEGATION_TOKEN"))) { LOG.info(applicationId + " found existing hdfs token " + token); hasHdfsToken = true; Text renewer = ((Token<AbstractDelegationTokenIdentifier>) token). decodeIdentifier().getRenewer(); if ((renewer != null && renewer.toString() .equals(Token.NON_RENEWER))) { continue; } } {code} - does conf.getStrings strip off the leading or ending empty strings? if not, we may strip those off. {code} String [] nns = conf.getStrings(MRJobConfig.JOB_NAMENODES_TOKEN_RENEWAL_EXCLUDE); {code} - given that this is a work-around fix, maybe not adding the NON_RENEWER publicly in common ? just check for null ? - Did you test the patch on real cluster ? > YARN's delegation-token handling disallows certain trust setups to operate > properly over DistCp > ----------------------------------------------------------------------------------------------- > > Key: YARN-3021 > URL: https://issues.apache.org/jira/browse/YARN-3021 > Project: Hadoop YARN > Issue Type: Bug > Components: security > Affects Versions: 2.3.0 > Reporter: Harsh J > Assignee: Yongjun Zhang > Attachments: YARN-3021.001.patch, YARN-3021.002.patch, > YARN-3021.003.patch, YARN-3021.004.patch, YARN-3021.patch > > > Consider this scenario of 3 realms: A, B and COMMON, where A trusts COMMON, > and B trusts COMMON (one way trusts both), and both A and B run HDFS + YARN > clusters. > Now if one logs in with a COMMON credential, and runs a job on A's YARN that > needs to access B's HDFS (such as a DistCp), the operation fails in the RM, > as it attempts a renewDelegationToken(…) synchronously during application > submission (to validate the managed token before it adds it to a scheduler > for automatic renewal). The call obviously fails cause B realm will not trust > A's credentials (here, the RM's principal is the renewer). > In the 1.x JobTracker the same call is present, but it is done asynchronously > and once the renewal attempt failed we simply ceased to schedule any further > attempts of renewals, rather than fail the job immediately. > We should change the logic such that we attempt the renewal but go easy on > the failure and skip the scheduling alone, rather than bubble back an error > to the client, failing the app submission. This way the old behaviour is > retained. -- This message was sent by Atlassian JIRA (v6.3.4#6332)