MiniDFSCluster hard-codes dfs.datanode.address to localhost
-----------------------------------------------------------
Key: HDFS-1875
URL: https://issues.apache.org/jira/browse/HDFS-1875
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Affects Versions: 0.22.0
Reporter: Eric Payne
Assignee: Eric Payne
Fix For: 0.23.0
When creating RPC addresses that represent the communication sockets for each
simulated DataNode, the MiniDFSCluster class hard-codes the address of the
dfs.datanode.address port to be "127.0.0.1:0"
The DataNodeCluster test tool uses the MiniDFSCluster class to create a
selected number of simulated datanodes on a single host. In the DataNodeCluster
setup, the NameNode is not simulated but is started as a separate daemon.
The problem is that if the write requrests into the simulated datanodes are
originated on a host that is not the same host running the simulated datanodes,
the connections are refused. This is because the RPC sockets that are started
by MiniDFSCluster are for "localhost" (127.0.0.1) and are not accessible from
outside that same machine.
It is proposed that the MiniDFSCluster.setupDatanodeAddress() method be
overloaded in order to accommodate an environment where the NameNode is on one
host, the client is on another host, and the simulated DataNodes are on yet
another host (or even multiple hosts simulating multiple DataNodes each).
The overloaded API would add a parameter that would be used as the basis for
creating the RPS sockets. By default, it would remain 127.0.0.1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
