[
https://issues.apache.org/jira/browse/HDFS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510273#comment-13510273
]
Chris Nauroth commented on HDFS-4269:
-------------------------------------
We merged this change to branch-trunk-win on Friday, 11/30. Unfortunately, this
had an unintended side effect of breaking on Windows, at least for single-node
developer setups, because of the code change to reject registration of an
unresolved data node:
{code}
public void registerDatanode(DatanodeRegistration nodeReg)
throws DisallowedDatanodeException {
InetAddress dnAddress = Server.getRemoteIp();
if (dnAddress != null) {
// Mostly called inside an RPC, update ip and peer hostname
String hostname = dnAddress.getHostName();
String ip = dnAddress.getHostAddress();
if (hostname.equals(ip)) {
LOG.warn("Unresolved datanode registration from " + ip);
throw new DisallowedDatanodeException(nodeReg);
}
{code}
On Windows, 127.0.0.1 does not resolve to localhost. It reports host name as
"127.0.0.1". Therefore, on Windows, running pseudo-distributed mode or
MiniDFSCluster-based tests always rejects the datanode registrations. (See
HADOOP-8414 for more discussion of the particulars of resolving 127.0.0.1 on
Windows.)
Potential fixes I can think of:
# Add special case logic to allow registration if ip.equals("127.0.0.1"). This
is the quick fix I applied to my dev environment to unblock myself last Friday.
# Add a check against NetUtils.getStaticResolution and register it with
NetUtils.addStaticResolution("127.0.0.1", "localhost") somewhere at
initialization time.
Below is a short code sample that demonstrates the problem. This is a very
rough approximation of the IPC Server/Connection and DatanodeManager logic.
When I run this server on Mac, it prints "connection from hostName = localhost,
hostAddress = 127.0.0.1, canonicalHostName = localhost" for any client
connection. On Windows, it prints "connection from hostName = 127.0.0.1,
hostAddress = 127.0.0.1, canonicalHostName = 127.0.0.1".
{code}
package cnauroth;
import java.io.PrintWriter;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.nio.channels.ServerSocketChannel;
class Main {
public static void main(String[] args) throws Exception {
ServerSocket ss = ServerSocketChannel.open().socket();
ss.bind(new InetSocketAddress("localhost", 1234), 0);
System.out.println("ss = " + ss);
for (;;) {
Socket s = ss.accept();
InetAddress addr = s.getInetAddress();
System.out.println("connection from hostName = " + addr.getHostName() +
", hostAddress = " + addr.getHostAddress() + ", canonicalHostName = " +
addr.getCanonicalHostName());
PrintWriter pw = new PrintWriter(s.getOutputStream());
pw.println("hello");
pw.close();
s.close();
}
}
}
{code}
> DatanodeManager#registerDatanode rejects all datanode registrations from
> localhost in single-node developer setup
> -----------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4269
> URL: https://issues.apache.org/jira/browse/HDFS-4269
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: trunk-win
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
>
> HDFS-3990 is a change that optimized some redundant DNS lookups. As part of
> that change, {{DatanodeManager#registerDatanode}} now rejects attempts to
> register a datanode for which the name has not been resolved. Unfortunately,
> this broke single-node developer setups on Windows, because Windows does not
> resolve 127.0.0.1 to "localhost".
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
