[
https://issues.apache.org/jira/browse/HDFS-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473472#comment-13473472
]
Karthik Kambatla commented on HDFS-4009:
----------------------------------------
Thanks Owen.
If I understand you correctly, in addition to letting user run commands/jobs as
another user, delegation tokens also act as a backup authentication mechanism,
should the kerberos ticket time out.
If that is the case, I see why we need to renew delegation tokens. In that
case, we need to decide if each FileSystem should have its own (instance of)
{{DelegationTokenRenewer}} or if we should have a single
{{DelegationTokenRenewerService}} that FileSystems can register/de-register. To
me, it seems reasonable to have a single service for different clients, by
augmenting the current {{DelegationTokenRenwer}}.
In my previous comment in the JIRA, I propose the following to that effect:
# Create a new abstract class TokenManager that holds a list (queue) of tokens,
their kind, renewal period, a TokenRenewer implementation, and an abstract
method #manage() to manage (renew/re-fetch/cancel) the tokens as appropriate.
# Each FileSystem (WebHdfs and Hftp) has its own implemenation (local extended
class) of the TokenManager.
# DelegationTokenRenewer should be a singleton.
# DelegationTokenRenewer allows registering and de-registering TokenManagers.
FileSystems call register() at init(), and de-register() at close().
# DelegationTokenRenewer stores the registered TokenManagers in its DelayQueue.
On a TokenManager's turn, the manage() method is invoked. If the manage fails,
the TokenManager is automatically de-registered.
Can you please weigh-in on this approach?
> WebHdfsFileSystem and HftpFileSystem don't need delegation tokens
> -----------------------------------------------------------------
>
> Key: HDFS-4009
> URL: https://issues.apache.org/jira/browse/HDFS-4009
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 2.0.0-alpha
> Reporter: Tom White
> Assignee: Karthik Kambatla
> Attachments: hadoop-8852.patch, hadoop-8852.patch,
> hadoop-8852-v1.patch
>
>
> Parent JIRA to track the work of removing delegation tokens from these
> filesystems.
> This JIRA has evolved from the initial issue of these filesystems not
> stopping the DelegationTokenRenewer thread they were creating.
> After further investigation, Daryn pointed out - "If you can get a token, you
> don't need a token"! Hence, these filesystems shouldn't use delegation tokens.
> Evolution of the JIRA is listed below:
> Update 2:
> DelegationTokenRenewer is not required. The filesystems that are using it
> already have Krb tickets and do not need tokens. Remove
> DelegationTokenRenewer and all the related logic from WebHdfs and Hftp
> filesystems.
> Update1:
> DelegationTokenRenewer should be Singleton - the instance and renewer threads
> should be created/started lazily. The filesystems using the renewer shouldn't
> need to explicity start/stop the renewer, and only register/de-register for
> token renewal.
> Initial issue:
> HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer
> thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
