Philip Zeyliger created HDFS-13797:
--------------------------------------
Summary: WebHDFS redirects to datanodes by hostname, ignoring bind
addresses
Key: HDFS-13797
URL: https://issues.apache.org/jira/browse/HDFS-13797
Project: Hadoop HDFS
Issue Type: Task
Components: webhdfs
Reporter: Philip Zeyliger
In testing Impala, we run datanodes bound to {{127.0.0.1}}. This works great,
except for WebHDFS, where the namenode redirects to a datanode by hostname,
leading to a "connection refused" error (since the datanode isn't listening).
Our workaround in Impala is to map hostname to {{127.0.0.1}} in {{/etc/hosts}}
which isn't so hot. (The workaround is at
https://github.com/apache/impala/blob/3334c167a69a6506d595898dff73fb16410266e6/bin/bootstrap_system.sh#L187
.)
I think there's a bug here. Reading
https://hadoop.apache.org/docs/r3.0.3/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html,
it looks like WebHDFS should perhaps respect
{{dfs.client.use.datanode.hostname}}, but I think it doesn't and always uses
the hostname. I think the relevant point where the namenode figures this out is
https://github.com/apache/hadoop/blob/ca20e0d7e9767a7362dddfea8ec19548947d3fd7/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java#L406
.
I recognize this is a non-production edge case: it makes no sense to listen
only on localhost except for testing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
