[
https://issues.apache.org/jira/browse/HADOOP-14920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16190180#comment-16190180
]
Xiaoyu Yao commented on HADOOP-14920:
-------------------------------------
Thanks [~xiaochen] for the quick responses.
bq. httpfs highly shares webhdfs code inside NN, and highly shares
authentication code with KMS. Would adding a service param here mean httpfs
would be double-service-configured?
webhdfs has its own delegation token handling before the refactoring work, so
the change in DelegationTokenAuthenticationHandler/DelegationTokenAuthenticator
is independent of webhdfs.
For services that use
DelegationTokenAuthenticationHandler/DelegationTokenAuthenticator like KMS:
*Java client always sets the service attribute for the returned token. No need
to add service parameter in the token request URL.
*Non-Java client usually can't set the service attribute for the returned token
easily. They can do that by adding service parameter to the token request URL
like what webhdfs supports with this patch.
In summary, we can only have one service attribute in the token. So double
service configuration won't be an issue here.
bq. if the service could be set arbitrarily, it would be good to find a way to
make it easier to debug...
In the patch attached, I've added the server side trace of all the parameters
in DelegationTokenManager#createToken(). On the client side,
KMSClientProvider#getActualUgi already dump all the UGI along with the all the
token infos such as kind, service, etc.
bq. Delegation token related ops are not even in the kms docs, would be great
to add those too. Can be a separate jira.
I've found the same issue and file a ticket HADOOP-12521 for myself. I will
spend some time to finish the doc after HADOOP-14920.
> KMSClientProvider won't work with KMS delegation token retrieved from
> non-Java client.
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-14920
> URL: https://issues.apache.org/jira/browse/HADOOP-14920
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Xiaoyu Yao
> Assignee: Xiaoyu Yao
> Attachments: HADOOP-14920.001.patch
>
>
> HADOOP-13381 added support to use KMS delegation token to connect to KMS
> server for key operations. However, the logic to check if the UGI container
> KMS delegation token assumes that the token must contain a service attribute.
> Otherwise, a KMS delegation token won't be recognized.
> For delegation token obtained via non-java client such curl (http), the
> default DelegationTokenAuthenticationHandler only support *renewer* parameter
> and assume the client itself will add the service attribute. This makes a
> java client with KMSClientProvdier can't use for KMS delegation token
> retrieved form non-java client because the token does not contain a service
> attribute.
> I did some investigation on this and found two solutions:
> 1. Similar use case exists for webhdfs, and webhdfs supports it with a
> ["service"
> parameter|https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Get_Delegation_Token].
> We can do this similarly by allowing client to specify a service attribute in
> the request URL and included in the token returned like webhdfs. Even though
> this will change in DelegationTokenAuthenticationHandler and may affect many
> other web component, this seems to be a clean and low risk solution because
> it will be an optional parameter. Also, other components get non-java client
> interop support for free if they have the similar use case.
> 2. The other way to solve this is to release the token check in
> KMSClientProvider to check only the token kind instead of the service. This
> is an easy work around but seems less optimal to me.
> cc: [~xiaochen] for additional input.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
