David,
For the Option #1.
I just changed the names to the IP Addresses, and it still comes up as
the external name and ip address in the log files, and on the job
tracker screen.
So option 1 is a no go.
When I change the "dfs.datanode.dns.interface" values it doesn't seem
to do anything. When I was search archived mail, this seemed to be a
the approach to change the NIC card being used for resolution. But
when I change it nothing happens, I even put in bogus values and still
no issues.
-John
On Jun 9, 2009, at 7:30 AM, David B. Ritch wrote:
I see several possibilities:
1) Use IP addresses instead of machine names in the config files.
2) Make sure that the names are in local /etc/hosts files, and that
they
resolve to the internal IP addresses.
3) Set up an internal name server, and make sure that it serves your
internal addresses and your cluster nodes point to it.
In general, how do you resolve your machine names to the internal
addresses?
David
John Martyniak wrote:
Hi,
I am creating a small Hadoop (0.19.1) cluster (2 nodes to start),
each
of the machines has 2 NIC cards (1 external facing, 1 internal
facing). It is important that Hadoop run and communicate on the
internal facing NIC (because the external facing NIC costs me money),
also the internal is more protected.
So I found a couple of parameters both in the mail archives and on
the
Hadoop core site, but they don't seem to do anything as the result is
always the same, it resolves to the external IP, and name (which is
in
DNS).
<property>
<name>dfs.datanode.dns.interface</name>
<value>en1</value>
<description>The name of the Network Interface from which a data
node should
report its IP address.
</description>
</property>
<property>
<name>dfs.datanode.dns.nameserver</name>
<value>DN</value>
<description>The host name or IP address of the name server (DNS)
which a DataNode should use to determine the host name used by the
NameNode for communication and display purposes.
</description>
</property>
<property>
<name>mapred.tasktracker.dns.interface</name>
<value>en1</value>
<description>The name of the Network Interface from which a task
tracker should report its IP address.
</description>
</property>
<property>
<name>mapred.tasktracker.dns.nameserver</name>
<value>DN</value>
<description>The host name or IP address of the name server (DNS)
which a TaskTracker should use to determine the host name used by
the JobTracker for communication and display purposes.
</description>
</property>
Any help would be greatly appreciated.
Thank you in advance.
-John
John Martyniak
Before Dawn Solutions, Inc.
9457 S. University Blvd #266
Highlands Ranch, CO 80126
o: 877-499-1562
c: 303-522-1756
e: j...@beforedawnsoutions.com
w: http://www.beforedawnsolutions.com
