Jason Lowe commented on YARN-3103:

bq.  Jason, does the patch work on secure cluster too? 

I didn't test this particular patch on a secure cluster, however I did test the 
same kind of change in MAPREDUCE-6230 on a secure cluster.  That patch did not 
work if I let it update the login user instead of the current user.  The 
current user is what the RPC layer is going to use (indeed, most of the purpose 
doAs exists is to specify which UGI the RPC layer will use), so I have no idea 
why we would try to circumvent that and update some other UGI.

bq. In MRAppMaster#initAndStartAppMaster, will it work if we insert the correct 
key/alias when adding credentials into appMasterUgi ?

Yes, I suppose there is one way to change what key/alias is associated with a 
token in Credentials and that's to create a complete copy of the credentials 
and specify the new alias when adding the token to the copy.  Since 
innitAndStartAppMaster is doing that, it could explicitly specify the key/alias 
by manually copying the credentials and special-casing the AMRM token.  Seems 
simpler to just re-use the alias set by the RM if that can work.  Or just add a 
UGI/Credentials API to update the service name of a token that also updates its 
key/alias in the credentials map so subsequent token stores will overwrite that 
token.  Or maybe a bit cleaner to have an API that explicitly says to replace 
one token with another, since the client can hunt down the old AMRM token using 
its updated service name.

> AMRMClientImpl does not update AMRM token properly
> --------------------------------------------------
>                 Key: YARN-3103
>                 URL: https://issues.apache.org/jira/browse/YARN-3103
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 2.6.0
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Blocker
>         Attachments: YARN-3103.001.patch
> AMRMClientImpl.updateAMRMToken updates the token service _before_ storing it 
> to the credentials, so the token is mapped using the newly updated service 
> rather than the empty service that was used when the RM created the original 
> AMRM token.  This leads to two AMRM tokens in the credentials and can still 
> fail if the AMRMTokenSelector picks the wrong one.
> In addition the AMRMClientImpl grabs the login user rather than the current 
> user when security is enabled, so it's likely the UGI being updated is not 
> the UGI that will be used when reconnecting to the RM.
> The end result is that AMs can fail with invalid token errors when trying to 
> reconnect to an RM after a new AMRM secret has been activated.

This message was sent by Atlassian JIRA

Reply via email to