[
https://issues.apache.org/jira/browse/YARN-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045062#comment-14045062
]
bc Wong commented on YARN-941:
------------------------------
Hi [~vinodkv], could you elaborate more on the point about master keys?
{quote}
Once we roll the master-keys, together with the fact that we want to support
services that run for ever, the only way we can support not expiring tokens is
by making ResourceManager remember master-keys for ever which is not feasible.
{quote}
The server side currently remembers the <id, password> pair for all valid
tokens. The master key isn't involved in verifying the token verification
process, from what I understand. So if we allow arbitrary expiration time for
AMRM tokens (to be specified by applications), then:
* RM needs to persist the <id, password> for all tokens that correspond to a
running application.
* Once an application finishes, RM can invalidate its token, and forget about
it.
* RM can keep rolling the master key, since it only affects how the "password"
(aka token secret) is generated.
Is my understanding of the password/master key interaction correct?
This "arbitrary expiration time" idea is conceptually a lot simpler than the
"token replacement patch" in this jira. So I feel it's worth a bit more
discussion. There are a few obvious attack vectors:
# *The attacker gains access to the persistence store, where the RM stores its
<id, password> map.* In this case, all bets are off. Neither solution is more
secure than the other.
# *The attacker snoops an insecure RPC channel and reads valid tokens from the
network.* The proper solution is to turn on RPC privacy. The token replacement
patch does not offer any real protection. On the contrary, it may give people a
_false sense of security_, which would be worse.
# *The attacker mounts a cryptographic attack, or somehow manages to guess a
valid <id, password> pair.* Token replacement is better because it limits the
exposure. But this attack is very unlikely. And we can counter that by using a
stronger hash function.
To me, the "arbitrary expiration time" approach is a lot simpler, without
compromising on security. What do you think?
> RM Should have a way to update the tokens it has for a running application
> --------------------------------------------------------------------------
>
> Key: YARN-941
> URL: https://issues.apache.org/jira/browse/YARN-941
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Robert Joseph Evans
> Assignee: Xuan Gong
> Attachments: YARN-941.preview.2.patch, YARN-941.preview.3.patch,
> YARN-941.preview.4.patch, YARN-941.preview.patch
>
>
> When an application is submitted to the RM it includes with it a set of
> tokens that the RM will renew on behalf of the application, that will be
> passed to the AM when the application is launched, and will be used when
> launching the application to access HDFS to download files on behalf of the
> application.
> For long lived applications/services these tokens can expire, and then the
> tokens that the AM has will be invalid, and the tokens that the RM had will
> also not work to launch a new AM.
> We need to provide an API that will allow the RM to replace the current
> tokens for this application with a new set. To avoid any real race issues, I
> think this API should be something that the AM calls, so that the client can
> connect to the AM with a new set of tokens it got using kerberos, then the AM
> can inform the RM of the new set of tokens and quickly update its tokens
> internally to use these new ones.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
