This is what I would consider a simple-enough, sane networking setup (and can assert that it works very well):
[NOTE: This is for simple, small clusters built by folks who are much too new to networking/haven't too much time. Of course, if you know what your DNS setup and resolution ought to look like, ignore this!] 1. A loopback address entry must exist in /etc/hosts. This must never be removed. "127.0.0.1 localhost.localdomain localhost" as the first line is an absolute, whether your services are going to utilize it, or not. 2. If you are looking at a small cluster and feel OK with just using /etc/hosts, then each of your hosts must be present in the /etc/hosts file used in the cluster. A line that goes "EXT.ERN.AL.IP host01.domain host01", repeated properly for each host in the cluster every node should know about (including itself, more importantly), must exist. 3. The (1) and (2) completes your /etc/hosts setup and may in the end look like this on ALL nodes, for an example (Yes, you may rsync/rdist it across): 127.0.0.1 localhost.localdomain localhost 5.1.0.1 node01.cluster node01 5.1.0.2 node02.cluster node02 5.1.0.3 node03.cluster node03 [NOTE: The IPs must come from the external NIC interface (eth0, etc.) address reported on each node via "ifconfig". I'll leave IP-assignment and DHCP usage outside of these guidelines.] 4. The /etc/nsswitch.conf must have, for its hosts entry, the config "hosts: files dns". This is usually the default - but ensure it is so on all nodes. 5. (3) and (4) done now makes sure that when resolution is demanded, the /etc/hosts file is what will be used, and that the /etc/hosts file is a good piece now. 6. Next step is to make sure that "hostname -f" and "hostname -s" report proper values on the whole cluster, for each node. The hostname of a machine is vital to be set to match the entry we refer to it as, via the /etc/hosts file. Know that the /etc/hosts file is a lookup file but the hostname comes from the OS itself, when self-queried by applications and tools. 7. For CentOS/RH/Fedora/etc. kinda distros, see http://www.electrictoolbox.com/changing-hostname-centos/ (File: /etc/sysconfig/network, config name HOSTNAME). For Ubuntu/Debian/etc. kinda distros, see http://www.ducea.com/2006/08/07/how-to-change-the-hostname-of-a-linux-system/ for one example (File: /etc/hostname, one line simple entry there). 8. Once the hostnames config on the OS match the corresponding node name defined in /etc/hosts, the "hostname -f" should, on node1, report "node01.cluster" and "hostname -s" should report "node01". 9. With (5) and (8) properly done now, stuff will work fine. Begin your Hadoop/HBase configs. HTH some folks building out their new, small clusters. I personally used bind9 on the first system I built, but I had way too much time then to sit down and debug whitespace issues :) On Fri, Sep 21, 2012 at 10:06 PM, Alberto Cordioli <[email protected]> wrote: > Ok. > I'm so sorry to tell you that probably HBase won't work!! :( > > Il giorno 21/set/2012 18.05, "Artem Ervits" <[email protected]> ha scritto: > >> I removed the reference to 127.0.0.1 from every node. Hadoop started as >> necessary and I didn't test hbase yet. >> >> >> >> Artem Ervits >> Data Analyst >> New York Presbyterian Hospital >> >> ----- Original Message ----- >> From: Alberto Cordioli [mailto:[email protected]] >> Sent: Friday, September 21, 2012 10:56 AM >> To: [email protected] <[email protected]> >> Subject: Re: Problem with Hadoop and /etc/hosts file >> >> Artem, it's the exact problem I have. >> How did you solve it? >> >> Alberto >> >> On 21 September 2012 14:18, Artem Ervits <[email protected]> wrote: >> > Actually, it is an hbase question. I faced same issue when I was testing >> recovery with hadoop 1 and I had a semi finished hbase cluster setup. The >> start up guide for hbase says to add 127.0.0.1 so when I did that on >> another node and started hadoop using that node as name node, hadoop would >> only see that node running on localhost and not on the static ip. The >> datanodes would not see the name node either. It is kind of confusing that >> hadoop wants it one way and hbase requires it the way hadoop won't work. >> > >> > >> > Artem Ervits >> > Data Analyst >> > New York Presbyterian Hospital >> > >> > ----- Original Message ----- >> > From: Stack [mailto:[email protected]] >> > Sent: Tuesday, September 18, 2012 01:08 PM >> > To: [email protected] <[email protected]> >> > Subject: Re: Problem with Hadoop and /etc/hosts file >> > >> > On Tue, Sep 18, 2012 at 12:01 AM, Alberto Cordioli >> > <[email protected]> wrote: >> >> Sorry, maybe I didn't explain well. >> >> I don't know hot to set up rDNS. I'd just know if this problem could >> >> generate the error I reported in the first post (since I get in any >> >> case the correct results). >> >> >> > >> > No need to apologize. I'm just suggesting that probably better lists >> > for figuring how to set up your networking. You could start with >> > this? http://en.wikipedia.org/wiki/Reverse_DNS_lookup >> > >> > St.Ack >> > >> > >> > -------------------- >> > >> > This electronic message is intended to be for the use only of the named >> recipient, and may contain information that is confidential or privileged. >> If you are not the intended recipient, you are hereby notified that any >> disclosure, copying, distribution or use of the contents of this message is >> strictly prohibited. If you have received this message in error or are not >> the named recipient, please notify us immediately by contacting the sender >> at the electronic mail address noted above, and delete and destroy all >> copies of this message. Thank you. >> > >> > >> > >> > >> > -------------------- >> > >> > This electronic message is intended to be for the use only of the named >> recipient, and may contain information that is confidential or privileged. >> If you are not the intended recipient, you are hereby notified that any >> disclosure, copying, distribution or use of the contents of this message is >> strictly prohibited. If you have received this message in error or are not >> the named recipient, please notify us immediately by contacting the sender >> at the electronic mail address noted above, and delete and destroy all >> copies of this message. Thank you. >> > >> > >> > >> >> >> >> -- >> Alberto Cordioli >> >> >> -------------------- >> >> This electronic message is intended to be for the use only of the named >> recipient, and may contain information that is confidential or privileged. >> If you are not the intended recipient, you are hereby notified that any >> disclosure, copying, distribution or use of the contents of this message is >> strictly prohibited. If you have received this message in error or are not >> the named recipient, please notify us immediately by contacting the sender >> at the electronic mail address noted above, and delete and destroy all >> copies of this message. Thank you. >> >> >> >> >> -------------------- >> >> This electronic message is intended to be for the use only of the named >> recipient, and may contain information that is confidential or privileged. >> If you are not the intended recipient, you are hereby notified that any >> disclosure, copying, distribution or use of the contents of this message is >> strictly prohibited. If you have received this message in error or are not >> the named recipient, please notify us immediately by contacting the sender >> at the electronic mail address noted above, and delete and destroy all >> copies of this message. Thank you. >> >> >> >> -- Harsh J
