Jian He commented on YARN-3055:

bq.  does it remove tokens from data structures in all cases or can a token get 
left in allTokens?
I think it does not have a leak. removeFailedDelegationToken will remove the 
token if renew fails. If there's a leak, the leak exists before  YARN-2704.
bq. The renewer looks like it may turn into a DOS weapon.
It does seem odd to get the expiration date by renewing the token. But there's 
just currently no way to get the expiration date other than the renew method.
bq. Any sub-job with the default of canceling tokens will kill the overall 
Am I missing something ? I think currently  the sub-job won't kill the overall 
workflow. the sub-job flag will be ignored, if the first job sets the flag.

Overall, I think overall the current patch will work, other than few comments I 
[~daryn], you mentioned you have another patch. could you share the patch or 
you think the current patch is fine ?

> The token is not renewed properly if it's shared by jobs (oozie) in 
> DelegationTokenRenewer
> ------------------------------------------------------------------------------------------
>                 Key: YARN-3055
>                 URL: https://issues.apache.org/jira/browse/YARN-3055
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: security
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>            Priority: Blocker
>         Attachments: YARN-3055.001.patch, YARN-3055.002.patch
> After YARN-2964, there is only one timer to renew the token if it's shared by 
> jobs. 
> In {{removeApplicationFromRenewal}}, when going to remove a token, and the 
> token is shared by other jobs, we will not cancel the token. 
> Meanwhile, we should not cancel the _timerTask_, also we should not remove it 
> from {{allTokens}}. Otherwise for the existing submitted applications which 
> share this token will not get renew any more, and for new submitted 
> applications which share this token, the token will be renew immediately.
> For example, we have 3 applications: app1, app2, app3. And they share the 
> token1. See following scenario:
> *1).* app1 is submitted firstly, then app2, and then app3. In this case, 
> there is only one token renewal timer for token1, and is scheduled when app1 
> is submitted
> *2).* app1 is finished, then the renewal timer is cancelled. token1 will not 
> be renewed any more, but app2 and app3 still use it, so there is problem.

This message was sent by Atlassian JIRA

Reply via email to