[
https://issues.apache.org/jira/browse/YARN-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346347#comment-14346347
]
Yongjun Zhang commented on YARN-3021:
-------------------------------------
Hi [~jianhe],
I agree with you about the longer term solution. However, with the short term
solution you suggested, #2 would let certain jobs continue to run even if they
would have failed token renewal without this short term solution.
Another alternative is,
# have MR client specify the token renewer it needs to use (instead of your
step 1), such as passing -Dmapreduce.job.delegation.tokenrenewer=null
# client code will update the token renewer if specified
# RM implements the logic to only renew its own token, however, this is
configurable by a new server-side config property, such as
yarn.resourcemanager.validate.tokenrenewer. This config property defaults to
false to retain old behavior (means not validating the renewer). We may
consider changing the default to true in the future.
This alternative avoids the new temporary API, but it also involves setting a
server-side config property that impacts all clients.
(The advantage of my earlier proposed solution has minimum impact, of course at
the cost of introducing the temporary new API)
Thoughts?
Thanks.
> YARN's delegation-token handling disallows certain trust setups to operate
> properly over DistCp
> -----------------------------------------------------------------------------------------------
>
> Key: YARN-3021
> URL: https://issues.apache.org/jira/browse/YARN-3021
> Project: Hadoop YARN
> Issue Type: Bug
> Components: security
> Affects Versions: 2.3.0
> Reporter: Harsh J
> Attachments: YARN-3021.001.patch, YARN-3021.002.patch,
> YARN-3021.003.patch, YARN-3021.patch
>
>
> Consider this scenario of 3 realms: A, B and COMMON, where A trusts COMMON,
> and B trusts COMMON (one way trusts both), and both A and B run HDFS + YARN
> clusters.
> Now if one logs in with a COMMON credential, and runs a job on A's YARN that
> needs to access B's HDFS (such as a DistCp), the operation fails in the RM,
> as it attempts a renewDelegationToken(…) synchronously during application
> submission (to validate the managed token before it adds it to a scheduler
> for automatic renewal). The call obviously fails cause B realm will not trust
> A's credentials (here, the RM's principal is the renewer).
> In the 1.x JobTracker the same call is present, but it is done asynchronously
> and once the renewal attempt failed we simply ceased to schedule any further
> attempts of renewals, rather than fail the job immediately.
> We should change the logic such that we attempt the renewal but go easy on
> the failure and skip the scheduling alone, rather than bubble back an error
> to the client, failing the app submission. This way the old behaviour is
> retained.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
