@Harsh Mind if fold the below into the refguide? St.Ack
On Fri, Sep 21, 2012 at 10:37 AM, Harsh J <[email protected]> wrote: > This is what I would consider a simple-enough, sane networking setup > (and can assert that it works very well): > > [NOTE: This is for simple, small clusters built by folks who are much > too new to networking/haven't too much time. Of course, if you know > what your DNS setup and resolution ought to look like, ignore this!] > > 1. A loopback address entry must exist in /etc/hosts. This must never > be removed. "127.0.0.1 localhost.localdomain localhost" as the first > line is an absolute, whether your services are going to utilize it, or > not. > > 2. If you are looking at a small cluster and feel OK with just using > /etc/hosts, then each of your hosts must be present in the /etc/hosts > file used in the cluster. A line that goes "EXT.ERN.AL.IP > host01.domain host01", repeated properly for each host in the cluster > every node should know about (including itself, more importantly), > must exist. > > 3. The (1) and (2) completes your /etc/hosts setup and may in the end > look like this on ALL nodes, for an example (Yes, you may rsync/rdist > it across): > > 127.0.0.1 localhost.localdomain localhost > 5.1.0.1 node01.cluster node01 > 5.1.0.2 node02.cluster node02 > 5.1.0.3 node03.cluster node03 > > [NOTE: The IPs must come from the external NIC interface (eth0, etc.) > address reported on each node via "ifconfig". I'll leave IP-assignment > and DHCP usage outside of these guidelines.] > > 4. The /etc/nsswitch.conf must have, for its hosts entry, the config > "hosts: files dns". This is usually the default - but ensure it is so > on all nodes. > > 5. (3) and (4) done now makes sure that when resolution is demanded, > the /etc/hosts file is what will be used, and that the /etc/hosts file > is a good piece now. > > 6. Next step is to make sure that "hostname -f" and "hostname -s" > report proper values on the whole cluster, for each node. The hostname > of a machine is vital to be set to match the entry we refer to it as, > via the /etc/hosts file. Know that the /etc/hosts file is a lookup > file but the hostname comes from the OS itself, when self-queried by > applications and tools. > > 7. For CentOS/RH/Fedora/etc. kinda distros, see > http://www.electrictoolbox.com/changing-hostname-centos/ (File: > /etc/sysconfig/network, config name HOSTNAME). For Ubuntu/Debian/etc. > kinda distros, see > http://www.ducea.com/2006/08/07/how-to-change-the-hostname-of-a-linux-system/ > for one example (File: /etc/hostname, one line simple entry there). > > 8. Once the hostnames config on the OS match the corresponding node > name defined in /etc/hosts, the "hostname -f" should, on node1, report > "node01.cluster" and "hostname -s" should report "node01". > > 9. With (5) and (8) properly done now, stuff will work fine. Begin > your Hadoop/HBase configs. > > HTH some folks building out their new, small clusters. I personally > used bind9 on the first system I built, but I had way too much time > then to sit down and debug whitespace issues :) > > On Fri, Sep 21, 2012 at 10:06 PM, Alberto Cordioli > <[email protected]> wrote: >> Ok. >> I'm so sorry to tell you that probably HBase won't work!! :( >> >> Il giorno 21/set/2012 18.05, "Artem Ervits" <[email protected]> ha scritto: >> >>> I removed the reference to 127.0.0.1 from every node. Hadoop started as >>> necessary and I didn't test hbase yet. >>> >>> >>> >>> Artem Ervits >>> Data Analyst >>> New York Presbyterian Hospital >>> >>> ----- Original Message ----- >>> From: Alberto Cordioli [mailto:[email protected]] >>> Sent: Friday, September 21, 2012 10:56 AM >>> To: [email protected] <[email protected]> >>> Subject: Re: Problem with Hadoop and /etc/hosts file >>> >>> Artem, it's the exact problem I have. >>> How did you solve it? >>> >>> Alberto >>> >>> On 21 September 2012 14:18, Artem Ervits <[email protected]> wrote: >>> > Actually, it is an hbase question. I faced same issue when I was testing >>> recovery with hadoop 1 and I had a semi finished hbase cluster setup. The >>> start up guide for hbase says to add 127.0.0.1 so when I did that on >>> another node and started hadoop using that node as name node, hadoop would >>> only see that node running on localhost and not on the static ip. The >>> datanodes would not see the name node either. It is kind of confusing that >>> hadoop wants it one way and hbase requires it the way hadoop won't work. >>> > >>> > >>> > Artem Ervits >>> > Data Analyst >>> > New York Presbyterian Hospital >>> > >>> > ----- Original Message ----- >>> > From: Stack [mailto:[email protected]] >>> > Sent: Tuesday, September 18, 2012 01:08 PM >>> > To: [email protected] <[email protected]> >>> > Subject: Re: Problem with Hadoop and /etc/hosts file >>> > >>> > On Tue, Sep 18, 2012 at 12:01 AM, Alberto Cordioli >>> > <[email protected]> wrote: >>> >> Sorry, maybe I didn't explain well. >>> >> I don't know hot to set up rDNS. I'd just know if this problem could >>> >> generate the error I reported in the first post (since I get in any >>> >> case the correct results). >>> >> >>> > >>> > No need to apologize. I'm just suggesting that probably better lists >>> > for figuring how to set up your networking. You could start with >>> > this? http://en.wikipedia.org/wiki/Reverse_DNS_lookup >>> > >>> > St.Ack >>> > >>> > >>> > -------------------- >>> > >>> > This electronic message is intended to be for the use only of the named >>> recipient, and may contain information that is confidential or privileged. >>> If you are not the intended recipient, you are hereby notified that any >>> disclosure, copying, distribution or use of the contents of this message is >>> strictly prohibited. If you have received this message in error or are not >>> the named recipient, please notify us immediately by contacting the sender >>> at the electronic mail address noted above, and delete and destroy all >>> copies of this message. Thank you. >>> > >>> > >>> > >>> > >>> > -------------------- >>> > >>> > This electronic message is intended to be for the use only of the named >>> recipient, and may contain information that is confidential or privileged. >>> If you are not the intended recipient, you are hereby notified that any >>> disclosure, copying, distribution or use of the contents of this message is >>> strictly prohibited. If you have received this message in error or are not >>> the named recipient, please notify us immediately by contacting the sender >>> at the electronic mail address noted above, and delete and destroy all >>> copies of this message. Thank you. >>> > >>> > >>> > >>> >>> >>> >>> -- >>> Alberto Cordioli >>> >>> >>> -------------------- >>> >>> This electronic message is intended to be for the use only of the named >>> recipient, and may contain information that is confidential or privileged. >>> If you are not the intended recipient, you are hereby notified that any >>> disclosure, copying, distribution or use of the contents of this message is >>> strictly prohibited. If you have received this message in error or are not >>> the named recipient, please notify us immediately by contacting the sender >>> at the electronic mail address noted above, and delete and destroy all >>> copies of this message. Thank you. >>> >>> >>> >>> >>> -------------------- >>> >>> This electronic message is intended to be for the use only of the named >>> recipient, and may contain information that is confidential or privileged. >>> If you are not the intended recipient, you are hereby notified that any >>> disclosure, copying, distribution or use of the contents of this message is >>> strictly prohibited. If you have received this message in error or are not >>> the named recipient, please notify us immediately by contacting the sender >>> at the electronic mail address noted above, and delete and destroy all >>> copies of this message. Thank you. >>> >>> >>> >>> > > > > -- > Harsh J
