[
https://issues.apache.org/jira/browse/HADOOP-3988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624392#action_12624392
]
Steve Loughran commented on HADOOP-3988:
----------------------------------------
1. The JVM does lots and lots of DNS caching of its own. To get it to cache
positive and negative DNS entries for less time than forever, you've got to
start the processes with a DNS TTL property, networkaddress.cache.ttl
By default, Java5 caches forever:
http://java.sun.com/j2se/1.5.0/docs/api/java/net/InetAddress.html
Java6 is a bit smarter, and only caches forever when a security manager is
installed
http://java.sun.com/javase/6/docs/technotes/guides/net/properties.html
You'd need to decide what is a good DNS cache TTL for the datanodes and push it
out to the scripts. I'm not 100% sure you can set this property inside the JVM
and have it taken up; the command line is the conventional way to do it.
2. The address of the namenode is currently set up in DataNode.startDataNode().
There are a few classes that assume that DataNode.getNameNodeAddr() is never
null; they'd need to change their assumptions.
3. DNS is slow and is one of the main delays in test runs right now.
What may work is leaving the current init code as it is, but whenever a
connection to the namenode fails, the datanode should re-read the namenode
address from the configuration and redo the nslookup; the scripts would need
patching to set a low TTL on the live systems. Rereading the address from the
configuration would be useful if the configuration was coming from something
like an LDAP server; you could change the hostname there and have it picked up
without DNS caching interfering.
> The elephant should remember names, not numbers.
> ------------------------------------------------
>
> Key: HADOOP-3988
> URL: https://issues.apache.org/jira/browse/HADOOP-3988
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.17.2
> Reporter: Allen Wittenauer
>
> The name node and the data node should not cache the resolution of host
> names, as doing so prevents the use of DNS CNAMEs for any sort of fail over
> capability.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
