[
https://issues.apache.org/jira/browse/YARN-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318891#comment-14318891
]
Vinod Kumar Vavilapalli commented on YARN-3021:
-----------------------------------------------
bq. Seems regardless of this jira, we could do a renewer address match as a
validation step. Right?
+1. Not as a validation, but to see if this RM should attempt renewal or not.
bq. In this case, actually looks like the renewer would be cluster A's yarn,
based on TokenCache@obtainTokensForNamenodesInternal and
Master.getMasterPrincipal.
bq. So it looks like that even if we check, the renewer would match in this
case. Please correct me if I'm wrong.
To make it work, we will still have to change the applications (MR etc). App
changes are needed irrespective of the approach.
bq. I'd be willing to accept that approach, but for one small worry: Any app
sending in a token with a bad renewer set could get through with such a change,
whereas previously it'd be rejected outright. Not that it'd be harmful (as it
is ignored), but it could still be seen as a behaviour change, no?
This is what you originally wanted :) ["In the 1.x JobTracker the same call is
present, but it is done asynchronously and once the renewal attempt failed we
simply ceased to schedule any further attempts of renewals, rather than fail
the job immediately."]
I think the problem is that RM doesn't have enough knowledge to know what is a
valid third-party renewer (that is not this RM itself), and what is an invalid
renewer. Even the app can really be not sure.
Overall I think "automatic token renewal" has always been an "auxiliary
service" provided by YARN's RM. If you want to make use of that service as an
application, you need to get token with the right token-service ('me') and pass
it to 'me' to renew it correctly. If either of those conditions, I'll not give
you that service.
Implicitly we also had a "automatic token validation" as a auxiliary feature.
But given the history I know, this was never our intention. The question is
whether we continue supporting this implicit aux feature or drop it. And given
my earlier point that RM cannot know either ways, this implicit feature was
always broken. I'm wary of adding this new API (I know I started with that
proposal :) )
> YARN's delegation-token handling disallows certain trust setups to operate
> properly over DistCp
> -----------------------------------------------------------------------------------------------
>
> Key: YARN-3021
> URL: https://issues.apache.org/jira/browse/YARN-3021
> Project: Hadoop YARN
> Issue Type: Bug
> Components: security
> Affects Versions: 2.3.0
> Reporter: Harsh J
> Attachments: YARN-3021.001.patch, YARN-3021.002.patch,
> YARN-3021.003.patch, YARN-3021.patch
>
>
> Consider this scenario of 3 realms: A, B and COMMON, where A trusts COMMON,
> and B trusts COMMON (one way trusts both), and both A and B run HDFS + YARN
> clusters.
> Now if one logs in with a COMMON credential, and runs a job on A's YARN that
> needs to access B's HDFS (such as a DistCp), the operation fails in the RM,
> as it attempts a renewDelegationToken(…) synchronously during application
> submission (to validate the managed token before it adds it to a scheduler
> for automatic renewal). The call obviously fails cause B realm will not trust
> A's credentials (here, the RM's principal is the renewer).
> In the 1.x JobTracker the same call is present, but it is done asynchronously
> and once the renewal attempt failed we simply ceased to schedule any further
> attempts of renewals, rather than fail the job immediately.
> We should change the logic such that we attempt the renewal but go easy on
> the failure and skip the scheduling alone, rather than bubble back an error
> to the client, failing the app submission. This way the old behaviour is
> retained.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
