[
https://issues.apache.org/jira/browse/HDFS-12621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212782#comment-16212782
]
Erik Krogen commented on HDFS-12621:
------------------------------------
Hm... There is some reference in the design document for HDFS-10467, but not a
lot of detail. As-is it actually will not work since it relies on an external
state store. This is what I had in mind for ViewFS, loosely based off of their
ideas:
* Upon a call to {{ViewFileSystem#getDelegationToken()}}, make a call to each
underlying Namenode. This is essentially the same as the behavior of
{{addDelegationToken()}}.
* Since {{getDelegationToken()}} can only store one delegation token, we need a
way to stuff all of the other tokens inside of it. Create a new
{{ViewFSTokenIdentifier extends TokenIdentifier}} which will stores all of them.
* When a {{ViewFileSystem}} method is called under a {{UserGroupInformation}}
which is authenticated via a {{ViewFSTokenIdentifier}}, extract the underlying
Token and send it along to the corresponding NN.
* To serialize {{ViewFSTokenIdentifier}} for later use by other clients, we
leverage the fact that all of the underlying Tokens are also serializable
({{Writable}}). The sequence of bytes that {{ViewFSTokenIdentifier}} serializes
to can simply be a concatenation of the serialized bytes for each underlying
Token, along with some information about which Token goes with which mount
point and/or NN URI. This should enable, after deserialization, any other
client to perform the same steps, assuming that it is using the same ViewFS
mount table.
It would require a fair amount of effort / code change, but all isolated to
within {{ViewFileSystem}} code.
> Inconsistency/confusion around ViewFileSystem.getDelagation
> ------------------------------------------------------------
>
> Key: HDFS-12621
> URL: https://issues.apache.org/jira/browse/HDFS-12621
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.7.3
> Reporter: Mohammad Kamrul Islam
> Assignee: Mohammad Kamrul Islam
>
> *Symptom*:
> When a user invokes ViewFileSystem.getDelegationToken(String renewer), she
> gets a "null". However, for any other file system, it returns a valid
> delegation token. For a normal user, it is very confusing and it takes
> substantial time to debug/find out an alternative.
> *Root Cause:*
> ViewFileSystem inherits the basic implementation from
> FileSystem.getDelegationToken() that returns "_null_". The comments in the
> source code indicates not to use it and instead use addDelegationTokens().
> However, it works fine DistributedFileSystem.
> In short, the same client call is working for hdfs:// but not for viewfs://.
> And there is no way of end-user to identify the root cause. This also creates
> a lot of confusion for any service that are supposed to work for both viewfs
> and hdfs.
> *Possible Solution*:
> _Option 1:_ Add a LOG.warn() with reasons/alternative before returning
> "null" in the base class.
> _Option 2:_ As done for other FS, ViewFileSystem can override the method with
> a implementation by returning the token related to fs.defaultFS. In this
> case, the defaultFS is something like "viewfs://..". We need to find out the
> actual namenode and uses that to retrieve the delegation token.
> _Option 3:_ Open for suggestion ?
> *Last note:* My hunch is : there are very few users who may be using
> viewfs:// with Kerberos. Therefore, it was not being exposed earlier.
> I'm working on a good solution. Please add your suggestion.
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
