[
https://issues.apache.org/jira/browse/HDFS-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568079#comment-13568079
]
Daryn Sharp commented on HDFS-4457:
-----------------------------------
Ok, I see what you're trying to do, but there are problems:
# The client should use the service as set by the server, esp. if it's going
through a proxy. The client thinks it contacted the proxy host for the token,
but the token as returned by the server will have the "real" host, so later the
client won't find the token when it uses the proxy.
# You can't assume how the token should be keyed in the Credentials. The key
is a private implementation detail. Guessing wrong will cause the job to fail
to submit when MR tries to get the token again.
-1 to the approach because I've invested a significant effort in ensuring that
0.20.5+ clients all set the service in the token. I paved the way for a
backwards-compatible change to token selection. I want to better support HA,
CNAMEs, multiple A-names, and multi-interface hosts. The server will set the
service to an opaque value and newer clients will use that for token selection,
but older clients will continue to set the service and work the same as they
ever did. We can also get rid of all the use_ip stuff. This patch would ruin
that feature...
I'd suggest using something like fetchdt that uses the standard token
acquisition apis that will populate the credentials for you. There is a
-tokenCacheFile in {{GenericOptionsParser}} that will allow the binary
credentials to be passed into a command. I've got a little patch somewhere
that reads the file into the UGI instead of just setting a mapred conf value.
If fetchdt, or something like it, isn't a reasonable approach, let's chat
offline to educate me on Hue.
> WebHDFS obtains/sets delegation token service hostname using wrong config
> leading to issues when NN is configured with 0.0.0.0 RPC IP
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4457
> URL: https://issues.apache.org/jira/browse/HDFS-4457
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 1.1.1, 2.0.2-alpha
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Priority: Critical
> Attachments: HDFS_4457.patch, HDFS_4457.patch
>
>
> If the NameNode RPC address is configured with an wildcard IP 0.0.0.0, then
> delegationotkens are configured with 0.0.0.0 as service and this breaks
> clients trying to use those tokens.
> Looking at NamenodeWebHdfsMethods#generateDelegationToken() the problem is
> SecurityUtil.setTokenService(t, namenode.getHttpAddress());, tracing back
> what is being used to resolve getHttpAddress() the NameNodeHttpServer is
> resolving the httpAddress doing a httpAddress = new
> InetSocketAddress(bindAddress.getAddress(), httpServer.getPort());
> , and if using "0.0.0.0" in the configuration, you get 0.0.0.0 from
> bindAddress.getAddress().
> Normally (non webhdfs) this is not an issue because it is the responsibility
> of the client, but in the case of WebHDFS, WebHDFS does it before returning
> the string version of the token (it must be this way because the client may
> not be a java client at all and cannot manipulate the DelegationToken as
> such).
> The solution (thanks to Eric Sammer for helping figure this out) is for
> WebHDFS to use the exacty hostname that came in the HTTP request as the
> service to set in the delegation tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
