[
https://issues.apache.org/jira/browse/YARN-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14370440#comment-14370440
]
Jian He edited comment on YARN-3021 at 3/20/15 12:29 AM:
---------------------------------------------------------
thanks Yongjun, some comments on the patch :
- DelegationTokenRenewer: the skipTokenRenewal check should be done under the
existing code {{if (token.getKind().equals(new
Text("HDFS_DELEGATION_TOKEN")))}} as below. And I think only doing this check
is enough, we don't need checks in other places.
{code}
if (token.isManaged()) {
if (token.getKind().equals(new Text("HDFS_DELEGATION_TOKEN"))) {
LOG.info(applicationId + " found existing hdfs token " + token);
hasHdfsToken = true;
Text renewer = ((Token<AbstractDelegationTokenIdentifier>) token).
decodeIdentifier().getRenewer();
if ((renewer != null && renewer.toString()
.equals(Token.NON_RENEWER))) {
continue;
}
}
{code}
- does conf.getStrings strip off the leading or ending empty strings? if not,
we may strip those off.
{code}
String [] nns =
conf.getStrings(MRJobConfig.JOB_NAMENODES_TOKEN_RENEWAL_EXCLUDE);
{code}
- given that this is a work-around fix, maybe not adding the NON_RENEWER
publicly in common ? just check for null ?
- Did you test the patch on real cluster ?
was (Author: jianhe):
thanks Yongjun, some comments on the patch !
- DelegationTokenRenewer: the skipTokenRenewal check should be done under the
existing code {{if (token.getKind().equals(new
Text("HDFS_DELEGATION_TOKEN")))}} as below. And I think only doing this check
is enough, we don't need checks in other places.
{code}
if (token.isManaged()) {
if (token.getKind().equals(new Text("HDFS_DELEGATION_TOKEN"))) {
LOG.info(applicationId + " found existing hdfs token " + token);
hasHdfsToken = true;
Text renewer = ((Token<AbstractDelegationTokenIdentifier>) token).
decodeIdentifier().getRenewer();
if ((renewer != null && renewer.toString()
.equals(Token.NON_RENEWER))) {
continue;
}
}
{code}
- does conf.getStrings strip off the leading or ending empty strings? if not,
we may strip those off.
{code}
String [] nns =
conf.getStrings(MRJobConfig.JOB_NAMENODES_TOKEN_RENEWAL_EXCLUDE);
{code}
- given that this is a work-around fix, maybe not adding the NON_RENEWER
publicly in common ? just check for null ?
- Did you test the patch on real cluster ?
> YARN's delegation-token handling disallows certain trust setups to operate
> properly over DistCp
> -----------------------------------------------------------------------------------------------
>
> Key: YARN-3021
> URL: https://issues.apache.org/jira/browse/YARN-3021
> Project: Hadoop YARN
> Issue Type: Bug
> Components: security
> Affects Versions: 2.3.0
> Reporter: Harsh J
> Assignee: Yongjun Zhang
> Attachments: YARN-3021.001.patch, YARN-3021.002.patch,
> YARN-3021.003.patch, YARN-3021.004.patch, YARN-3021.patch
>
>
> Consider this scenario of 3 realms: A, B and COMMON, where A trusts COMMON,
> and B trusts COMMON (one way trusts both), and both A and B run HDFS + YARN
> clusters.
> Now if one logs in with a COMMON credential, and runs a job on A's YARN that
> needs to access B's HDFS (such as a DistCp), the operation fails in the RM,
> as it attempts a renewDelegationToken(…) synchronously during application
> submission (to validate the managed token before it adds it to a scheduler
> for automatic renewal). The call obviously fails cause B realm will not trust
> A's credentials (here, the RM's principal is the renewer).
> In the 1.x JobTracker the same call is present, but it is done asynchronously
> and once the renewal attempt failed we simply ceased to schedule any further
> attempts of renewals, rather than fail the job immediately.
> We should change the logic such that we attempt the renewal but go easy on
> the failure and skip the scheduling alone, rather than bubble back an error
> to the client, failing the app submission. This way the old behaviour is
> retained.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
