[
https://issues.apache.org/jira/browse/HDFS-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569183#comment-13569183
]
Daryn Sharp commented on HDFS-4457:
-----------------------------------
Ignoring the issue of the client relying on the server to set the service -
which is what I don't approve of since it's a big step backwards - you still
have the problem if a proxy is between the client and the webhdfs server. The
token will contain the hostname that the proxy used to contact the server, not
the hostname the client used to contact the proxy. The proxy, or even some
form of NAT may be redirecting the port. The server doesn't know this, only
the client knows what port it thinks it contacted.
The remote server also doesn't have the ability to know if the client has
use_ip enabled or disabled.
Basically, only the client that requested the token knows the exact host:port
authority it used to request the token. When it attempts to re-contact that
service, it needs to match the service with the authority.
My second concern is that you must be assuming the key to store the token in
the credentials. It currently happens to be the token's service, but it's a
private implementation detail. If the key format changes, and the passed-along
token is added to the credentials with the old format, then job submission will
attempt to reacquire the token and fail. Fetchdt solves this by allowing you
to acquire tokens and opaquely pass them along in binary form.
What error are you encountering with fetchdt? It's working for me on a
production cluster:
{noformat}
$ hdfs fetchdt -fs webhdfs://host /tmp/tokens
Fetched token for host:50070 into file:/tmp/tokens
{noformat}
> WebHDFS obtains/sets delegation token service hostname using wrong config
> leading to issues when NN is configured with 0.0.0.0 RPC IP
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4457
> URL: https://issues.apache.org/jira/browse/HDFS-4457
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 1.1.1, 2.0.2-alpha
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Priority: Critical
> Attachments: HDFS_4457.patch, HDFS_4457.patch
>
>
> If the NameNode RPC address is configured with an wildcard IP 0.0.0.0, then
> delegationotkens are configured with 0.0.0.0 as service and this breaks
> clients trying to use those tokens.
> Looking at NamenodeWebHdfsMethods#generateDelegationToken() the problem is
> SecurityUtil.setTokenService(t, namenode.getHttpAddress());, tracing back
> what is being used to resolve getHttpAddress() the NameNodeHttpServer is
> resolving the httpAddress doing a httpAddress = new
> InetSocketAddress(bindAddress.getAddress(), httpServer.getPort());
> , and if using "0.0.0.0" in the configuration, you get 0.0.0.0 from
> bindAddress.getAddress().
> Normally (non webhdfs) this is not an issue because it is the responsibility
> of the client, but in the case of WebHDFS, WebHDFS does it before returning
> the string version of the token (it must be this way because the client may
> not be a java client at all and cannot manipulate the DelegationToken as
> such).
> The solution (thanks to Eric Sammer for helping figure this out) is for
> WebHDFS to use the exacty hostname that came in the HTTP request as the
> service to set in the delegation tokens.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
