[
https://issues.apache.org/jira/browse/HDFS-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568111#comment-13568111
]
Alejandro Abdelnur commented on HDFS-4457:
------------------------------------------
Daryn,
Regarding "The client should use the service as set by the server,", this is
not true today, DFSClient.getDelegationToken() &
WebHdfsFileSystem.getDelegationToken() sets the service HOST:PORT when it
receives the token using the HOST:PORT used by the client. The token fetched by
a client should have the HOST:PORT the client used to access the service
(HDFS/WEBHDFS), if the service is being offered with multihoming, the service
cannot know/assume what HOST are visible to the client, unless the it has a
hint from the client (this is what this patch is done for WebHDFS).
Regarding "You can't assume...", I don't think there is any assumption here,
the client is getting a blob and passes that blob to another client, that is
the whole purpose of delegation tokens, no? This is using the built in
mechanism in MapReduce to populate the token cache with the contents of the
binary representation of the token. This ends up being used to submit the job,
thus in the credentials of the alternate client.
What this patch does, is to enable non java clients that request a delegation
token from WebHDFS to have a delegation token ready to be handed to another
client.
This patch does not affect in any maner java clients that request a delegation
token as they continue injecting the service when they receive the delegation
token.
Oozie does not have this problem because Oozie uses the java client and gets
the service injection for free. Hue in the other hand, because not being Java,
cannot do this.
Regarding fetchdt, it does not seem to fetch tokens of WebHDFS.
Finally, in the scenario the approach I'm proposing does not cut it, we still
have problem with WebHDFS using setting the IP wildcard of the NN RPC
configuration.
Thoughts?
> WebHDFS obtains/sets delegation token service hostname using wrong config
> leading to issues when NN is configured with 0.0.0.0 RPC IP
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4457
> URL: https://issues.apache.org/jira/browse/HDFS-4457
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 1.1.1, 2.0.2-alpha
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Priority: Critical
> Attachments: HDFS_4457.patch, HDFS_4457.patch
>
>
> If the NameNode RPC address is configured with an wildcard IP 0.0.0.0, then
> delegationotkens are configured with 0.0.0.0 as service and this breaks
> clients trying to use those tokens.
> Looking at NamenodeWebHdfsMethods#generateDelegationToken() the problem is
> SecurityUtil.setTokenService(t, namenode.getHttpAddress());, tracing back
> what is being used to resolve getHttpAddress() the NameNodeHttpServer is
> resolving the httpAddress doing a httpAddress = new
> InetSocketAddress(bindAddress.getAddress(), httpServer.getPort());
> , and if using "0.0.0.0" in the configuration, you get 0.0.0.0 from
> bindAddress.getAddress().
> Normally (non webhdfs) this is not an issue because it is the responsibility
> of the client, but in the case of WebHDFS, WebHDFS does it before returning
> the string version of the token (it must be this way because the client may
> not be a java client at all and cannot manipulate the DelegationToken as
> such).
> The solution (thanks to Eric Sammer for helping figure this out) is for
> WebHDFS to use the exacty hostname that came in the HTTP request as the
> service to set in the delegation tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
