[
https://issues.apache.org/jira/browse/HADOOP-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626915#action_12626915
]
Steve Loughran commented on HADOOP-3426:
----------------------------------------
>> cluster without DNS
>IP address should be used without the hostname
OK. One thing we can't rely on here is any kind of rDNS to work, hostname
lookup may time out, etc.
>> cluster on xen/vmware without DNS
>Same as above.
OK
>> ... without internet
>You mean without network set up? If we can detect it, then it should be a
>warning and we should use the localhost interface if that exists. If not it
>should be a hard error.
I was thinking a little private xen/vmware virtual cluster with no external
connectivity or DNS. Its a self-contained network.
>> developer laptop on the move running tests/demos
>may be need not be handled now.
Makes sense.
One thing that is hard when handling different network setups is actually
creating the relevant network configurations. Most developers work from well
managed networks where things like DNS work properly; its out in the
mass-end-user world that networking becomes atrocious. I think for Hadoop, it
would be important to list the supported/unsupported network configurations,
and only target the supported stuff. The unsupported stuff -if it can be
detected and warned about, then all is well, but some requirements "you have a
network and it is fairly reliable" may be things that we can just rely on, and
opt not to feel guilty if the code doesn't work when that requirement isn't met.
> Datanode does not start up if the local machines DNS isnt working right and
> dfs.datanode.dns.interface==default
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3426
> URL: https://issues.apache.org/jira/browse/HADOOP-3426
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Environment: Ubuntu 8.04, at home, no reverse DNS
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Attachments: hadoop-3426.patch, hadoop-3426.patch
>
>
> This is the third Java project I've been involved in that doesnt work on my
> home network, due to implementation issues with
> java.net.InetAddress.getLocalHost(), issues that only show up on an unamanged
> network. Fortunately my home network exists to find these problems early.
> In hadoop, if the local hostname doesnt resolve, the datanode does not start
> up:
> Caused by: java.net.UnknownHostException: k2: k2
> at java.net.InetAddress.getLocalHost(InetAddress.java:1353)
> at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:185)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:184)
> at org.apache.hadoop.dfs.DataNode.(DataNode.java:162)
> at org.apache.hadoop.dfs.ExtDataNode.(ExtDataNode.java:55)
> at
> org.smartfrog.services.hadoop.components.datanode.DatanodeImpl.sfStart(DatanodeImpl.java:60)
> While this is a valid option in a production (non-virtual) cluster, if you
> are playing with VMWare/Xen private networks or on a home network, you can't
> rely on DNS.
> 1. In these situations, its usually better to fall back to using "localhost"
> or 127.0.0.1 as a hostname if Java can't work it out for itself,
> 2. Its often good to cache this if used in lots of parts of the system,
> otherwise the 30s timeouts can cause problems of their own.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
