Hi, You can't do that.
The problem is that Hadoop is going to pick up your external IP address because that's what the machine name resolves to. Then your slave nodes are on the internal route and you don't see them. Is it a bug? Maybe. More like a design defect. The work around is to forget about using the second nic card for hadoop/hbase traffic. Or make the internal network match your machine name and its dns information. Then use the second ip address to communicate with the outside world. So if your machine name is foo, and your dominiant ip address is on eth(0), you want foo.company.com to resolve to eth(0) and foo-ext.company.com to resolve to eth(1). Its backwards but it should work. IMHO, after looking at this issue, it really doesn't matter since the cloud shouldn't be getting a lot of external traffic except on the name node/job tracker nodes which could be multi-homed. HTH -Mike PS. If I'm wrong please let me know. I had this problem and at the time, no solution was available. > Subject: DNS.reverseDns() fallback > From: [email protected] > Date: Fri, 28 May 2010 12:24:12 +0200 > To: [email protected] > > Hi guys, > > i have following situation. A network setup with 2 network interfaces, eth0 > (external) and eth1(internal). > Now in order to use the internal ips for hadoop i set > dfs.datanode.dns.interface and mapred.tasktracker.dns.interface to eth1. > When tasktracker or datanode starting the following happens: > - hadoop picks the ip(s) from eth1 > - hadoop tries to do a reverse dns lookup on the internal ip(s) - which fails > - hadoop fallback to InetAddress.getLocalHost() - which is the external dns > > So in the end DNS.getDefaultIp(eth1) returns the internal ip whereas > DNS.getDefaultHost(eth1) returns the external dns. > > Isn't this a bug. Should in case the reverse lookup fails > InetAddress.getByName(ips[ctr]).getCanonicalHost() rather be taken ? > Are there any suggested workarounds for this situation ? > > best regards > Johannes > _________________________________________________________________ Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1
