bc Wong commented on YARN-941:

I'm fine with [~xgong]'s solution. I'd still like to see something more generic 
to make tokens (HDFS token, HBase token, etc) work with long running apps 
though. Perhaps I'll pursue the "arbitrary expiration time" approach in another 

RPC privacy is a very expensive solution for AM-RM communication. First, it 
needs setup so AM/RM have access to key infrastructure - having this burden on 
all applications is not reasonable. This is compounded by the fact that we use 
AMRMTokens in non-secure mode too. Second, AM - RM communication is a very 
chatty protocol, it's likely the overhead is huge..

True security is often costly. The web/consumer industry went through the same 
exercise with HTTP vs HTTPS. You can get at least 10x better performance with 
HTTP. But in the end, everybody decided that it's worth it. And passing tokens 
around without RPC privacy is just like sending passwords around on HTTP 
without SSL.

Unfortunately with long running services (the focus of this JIRA), this attack 
and its success is not as unlikely. This is the very reason why we roll 
master-keys every so often in the first place.

With the rolling master key, it's unlikely for the attack to gather enough 
cipher text to mount that attack. Besides, a longer key would require so much 
computation to attack that it'd be infeasible.

Anyway, appreciate your response, and I'll follow up in another jira.

> RM Should have a way to update the tokens it has for a running application
> --------------------------------------------------------------------------
>                 Key: YARN-941
>                 URL: https://issues.apache.org/jira/browse/YARN-941
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Robert Joseph Evans
>            Assignee: Xuan Gong
>         Attachments: YARN-941.preview.2.patch, YARN-941.preview.3.patch, 
> YARN-941.preview.4.patch, YARN-941.preview.patch
> When an application is submitted to the RM it includes with it a set of 
> tokens that the RM will renew on behalf of the application, that will be 
> passed to the AM when the application is launched, and will be used when 
> launching the application to access HDFS to download files on behalf of the 
> application.
> For long lived applications/services these tokens can expire, and then the 
> tokens that the AM has will be invalid, and the tokens that the RM had will 
> also not work to launch a new AM.
> We need to provide an API that will allow the RM to replace the current 
> tokens for this application with a new set.  To avoid any real race issues, I 
> think this API should be something that the AM calls, so that the client can 
> connect to the AM with a new set of tokens it got using kerberos, then the AM 
> can inform the RM of the new set of tokens and quickly update its tokens 
> internally to use these new ones.

This message was sent by Atlassian JIRA

Reply via email to