bc Wong commented on YARN-941:

Hi [~vinodkv], could you elaborate more on the point about master keys?
Once we roll the master-keys, together with the fact that we want to support 
services that run for ever, the only way we can support not expiring tokens is 
by making ResourceManager remember master-keys for ever which is not feasible.

The server side currently remembers the <id, password> pair for all valid 
tokens. The master key isn't involved in verifying the token verification 
process, from what I understand. So if we allow arbitrary expiration time for 
AMRM tokens (to be specified by applications), then:
* RM needs to persist the <id, password> for all tokens that correspond to a 
running application.
* Once an application finishes, RM can invalidate its token, and forget about 
* RM can keep rolling the master key, since it only affects how the "password" 
(aka token secret) is generated.

Is my understanding of the password/master key interaction correct?

This "arbitrary expiration time" idea is conceptually a lot simpler than the 
"token replacement patch" in this jira. So I feel it's worth a bit more 
discussion. There are a few obvious attack vectors:
# *The attacker gains access to the persistence store, where the RM stores its 
<id, password> map.* In this case, all bets are off. Neither solution is more 
secure than the other.
# *The attacker snoops an insecure RPC channel and reads valid tokens from the 
network.* The proper solution is to turn on RPC privacy. The token replacement 
patch does not offer any real protection. On the contrary, it may give people a 
_false sense of security_, which would be worse.
# *The attacker mounts a cryptographic attack, or somehow manages to guess a 
valid <id, password> pair.* Token replacement is better because it limits the 
exposure. But this attack is very unlikely. And we can counter that by using a 
stronger hash function.

To me, the "arbitrary expiration time" approach is a lot simpler, without 
compromising on security. What do you think?

> RM Should have a way to update the tokens it has for a running application
> --------------------------------------------------------------------------
>                 Key: YARN-941
>                 URL: https://issues.apache.org/jira/browse/YARN-941
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Robert Joseph Evans
>            Assignee: Xuan Gong
>         Attachments: YARN-941.preview.2.patch, YARN-941.preview.3.patch, 
> YARN-941.preview.4.patch, YARN-941.preview.patch
> When an application is submitted to the RM it includes with it a set of 
> tokens that the RM will renew on behalf of the application, that will be 
> passed to the AM when the application is launched, and will be used when 
> launching the application to access HDFS to download files on behalf of the 
> application.
> For long lived applications/services these tokens can expire, and then the 
> tokens that the AM has will be invalid, and the tokens that the RM had will 
> also not work to launch a new AM.
> We need to provide an API that will allow the RM to replace the current 
> tokens for this application with a new set.  To avoid any real race issues, I 
> think this API should be something that the AM calls, so that the client can 
> connect to the AM with a new set of tokens it got using kerberos, then the AM 
> can inform the RM of the new set of tokens and quickly update its tokens 
> internally to use these new ones.

This message was sent by Atlassian JIRA

Reply via email to