[
https://issues.apache.org/jira/browse/HADOOP-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179643#comment-13179643
]
Daryn Sharp commented on HADOOP-7933:
-------------------------------------
{quote}
bq. It appears Credentials was selected for convenience since that's what the
MR's TokenCache uses internally? It appears Credentials was selected for
convenience since that's what the MR's TokenCache uses internally?
The API is limited private (at least for now) to MR and HDFS - so I don't see
this as a very big issue. Changing it to a set is a simple change though.
Thoughts ?
{quote}
Although it's limited audience, it's arguably creating a cross-component
dependency where the cart is leading the horse. However, {{Credentials}} is
part of common, so perhaps it's not a big deal or even moot if the next point
is addressed.
{quote}
bq. I'm concerned that this appears to be creating a cross-component dependency
whereby ViewFileSystem it required to "assume" how MR's TokenCache will want
the tokens keyed within its Credentials.
I'm not very happy with this either - suggestion on a better approach ? Again,
being LimitedPrivate to MR and HDFS limits the impact.
{quote}
A simple solution is to centralize the chosen key to a single location. Awhile
back I played with adding {{Credentials#addToken(Token)}} or
{{Credentials#addServiceToken(token)}} to enforce {{token.getService()}} as the
key. Along the same lines, {{Credentials#getToken(TokenIssuer)}} could fetch
all tokens for the token issuing object.
The cited inconsistency about {{getDelegationTokens}} was a misread of the
code...
> Viewfs changes for MAPREDUCE-3529
> ---------------------------------
>
> Key: HADOOP-7933
> URL: https://issues.apache.org/jira/browse/HADOOP-7933
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: HADOOP-7933.txt, HADOOP7933_v1.txt, HADOOP7933_v2.txt,
> HDFS2665_v1.txt, HDFS2665_v1.txt
>
>
> ViewFs.getDelegationTokens returns a list of tokens for the associated
> namenodes. Credentials serializes these tokens using the service name for the
> actual namenodes. Effectively, tokens are not cached for viewfs (some more
> details in MR 3529). Affects any job which uses the TokenCache in tasks along
> with viewfs (some Pig jobs).
> Talk to Jitendra about this, some options
> 1. Change Credentials.getAllTokens to return the key, instead of just a token
> list (associate the viewfs canonical name with a token in credentials)
> 2. Have viewfs issue a fake token.
> Both of these would allow for a single viewfs configuration only.
> 3. An additional API in FileSystem - something like
> getDelegationTokens(String renewer, Credentials credentials) - which would
> check the credentials object before making token requests to the actual
> namenode.
> 4. An additional API in FileSystem - getCanonicalServiceNames - similar to
> getDelegationTokens, which would return service names for the actual
> namenodes. TokenCache/Credentials can work using this list.
> 5. have getDelegationTokens check the current UGI - and fetch tokens only if
> they don't exist.
> Have a quick patch for 3, along with associated MR changes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
