Jian He commented on YARN-2704:

Thanks Vinod for the review !
bq. removeApplicationFromRenewal() is only called when log-aggregation is 
enabled, so that will affect the new credentials map?
removeApplicationFromRenewal is actually directly called when log-aggregation 
is disabled. if it’s enabled, they are added to a delayed map.
bq. Just bubble up the IOException instead of wrapping it in 
It’s inside the run method, so I wrap it with runTimeException
bq. We don't need to renew the token immediately after obtaining it?
This is to get the expiration date, token itself doesn’t have the expiration 
bq. Make 36000000 a constant. And why is it 10 hours? Shouldn't this be a 
function of the max-life-time in general?
This guarantees we have 10h minimum buffer to distribute the tokens. Any time 
more that is not necessary ?
bq. we should also look for the service name matching the default-file-system.
there’s no easy way to get the service name base on the file-system object, and 
the hdfs token service-name varies in different case: e.g. ha/non-ha; 
bq. The log message " found existing hdfs token " needs to be a debug log
Regarding the info/debug level logs,  these logs are all low frequency logs, by 
default it’s only 1 day a time(renew interval) . And it’s so much easier to 
debug in info level than debug level. maybe in info level while stablizing this 

>  Localization and log-aggregation will fail if hdfs delegation token expired 
> after token-max-life-time
> ------------------------------------------------------------------------------------------------------
>                 Key: YARN-2704
>                 URL: https://issues.apache.org/jira/browse/YARN-2704
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Jian He
>            Assignee: Jian He
>         Attachments: YARN-2704.1.patch
> In secure mode, YARN requires the hdfs-delegation token to do localization 
> and log aggregation on behalf of the user. But the hdfs delegation token will 
> eventually expire after max-token-life-time.  So,  localization and log 
> aggregation will fail after the token expires.

This message was sent by Atlassian JIRA

Reply via email to