I have this in hbase-site.xml:
<property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:9000/hbase</value>
<description>The directory shared by region servers.
Should be fully-qualified to include the filesystem to use.
E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
</description>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
<description>For psuedo-distributed, you want to set this to true.
false means that HBase tries to put Master + RegionServers in one
process.
Pseudo-distributed = seperate processes/pids</description>
</property>
<property>
<name>hbase.regionserver.hlog.replication</name>
<value>1</value>
<description>For HBase to offer good data durability, we roll logs if
filesystem replication falls below a certain amount. In
psuedo-distributed
mode, you normally only have the local filesystem or 1 HDFS DataNode, so
you
don't want to roll logs constantly.</description>
</property>
<property>
<name>hbase.tmp.dir</name>
<value>/tmp/hbase-testing</value>
<description>Temporary directory on the local filesystem.</description>
</property>
I also hase Hadoop conf directory in HBASE_CLASSPATH (hbase-env.sh).
I just tried etc/hosts with "127.0.0.1 localhost.localdomain
localhost" line. I got the same error I was getting before. I switched it
back to "127.0.0.1 localhost" and it worked. In between those changes,
I stopped hbase, hadoop and killed still running region server. I hope that
helps.
N.Gesli
On Thu, Sep 16, 2010 at 7:04 AM, Michael Scott <[email protected]> wrote:
> This sounds promising, I have one quick question about your steps: where
> in
> the Hbase config *site*.xml did you make the change back to localhost? My
> hbase master is using the public IP address (97.86.88.18), and I don't
> think
> I've told it to. I want to convince hbase to get rid of the line in the
> log
> file that says something like:
>
> 2010-09-16 09:59:21,727 INFO org.apache.hadoop.hbase.master.HMaster: My
> address is 97-86-88-18.static.aldl.mi.charter.com:60000
>
> (Note that my /etc/hosts has only the one line
> 127.0.0.1 localhost.localdomain localhost
> since I'm not running ipv6, but somehow hbase knows that the interface is a
> comcast static address. I can use /etc/hosts to change that to the
> registered domain name for 97-86-88-18, but this doesn't help.)
>
> To reply to Ryan's question, my ifconfig gives:
>
> eth0 Link encap:Ethernet HWaddr 00:24:E8:01:DA:B8
> inet addr:10.0.0.2 Bcast:10.0.0.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:319475 errors:0 dropped:0 overruns:0 frame:0
> TX packets:290698 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:108186958 (103.1 MiB) TX bytes:187845633 (179.1 MiB)
> Interrupt:28 Base address:0xa000
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:370795 errors:0 dropped:0 overruns:0 frame:0
> TX packets:370795 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:108117402 (103.1 MiB) TX bytes:108117402 (103.1 MiB)
>
> Thanks a bunch!
>
> Michael
>
> On Thu, Sep 16, 2010 at 1:12 AM, N.N. Gesli <[email protected]> wrote:
>
> > Hi Michael,
> >
> > I was having a similar problem and following this thread for any
> > suggestions. I tried everything suggested and more.
> >
> > I was trying to run Hadoop/Hbase pseudo distributed version on my Mac. I
> > initially started with Hadoop 21.0 and Hbase 0.89 versions. I had exactly
> > the same error that you were getting. Then switched to Hadoop 20.2 and
> > Hbase
> > 20.6 - still HMaster was not starting. Then finally it worked. Below are
> my
> > steps to success :)
> >
> > * stopped hbase
> > * stopped hadoop
> > * run jps; RegionServer was still running; killed it manually
> > * in tmp directory (where hadoop namenode and *.pid files are stored) I
> > removed everything related to hadoop and hbase, including the
> directories.
> > (I had no data in Hadoop, so I could do this)
> > * changed the ports back to default 600**
> > * changed back Hadoop and Hbase configurations to "localhost" in
> *site*.xml
> > and regionservers. (Only I will be using this - no remote connection)
> > * changed back my /etc/hosts to the original version. It looks like this:
> > 127.0.0.1 localhost
> > ::1 localhost
> > fe80::1%lo0 localhost
> > * reformatted the Hadoop namenode
> > * started Hadoop
> > * started HBase and it worked :)
> >
> > Let me know if you want to know any specific configuration.
> >
> > N.Gesli
> >
> > On Wed, Sep 15, 2010 at 10:41 PM, Ryan Rawson <[email protected]>
> wrote:
> >
> > > What is your ifconfig output looking like?
> > >
> > >
> > >
> > > On Wed, Sep 15, 2010 at 10:07 PM, Michael Scott <[email protected]>
> > > wrote:
> > > > Thanks for the continued advice. I am still confused by the
> different
> > > > behaviors of hadoop and hbase. As I said before, I can't get hbase to
> > > work
> > > > on any of the ports that hadoop works on, so I guess hadoop and hbase
> > are
> > > > using different interfaces. Why is this, and can't I ask hbase to
> use
> > > the
> > > > interface that hadoop uses? What interfaces are hadoop and hbase
> > using?
> > > >
> > > > Also (and maybe this is the wrong forum for this question), how can I
> > get
> > > my
> > > > OS to allow me to open 60000 using the IP address? I have
> temporarily
> > > > disabled selinux and iptables, as I thought that this would simply
> > allow
> > > all
> > > > port connections. Still, this works just fine:
> > > > bash-4.0$ nc -l 60000 > /tmp/nc.out
> > > >
> > > > but this does not:
> > > > bash-4.0$ nc -l 97.86.88.18 60000 > /tmp/nc.out
> > > > (returns "nc: Cannot assign requested address"; I get the same error
> > for
> > > the
> > > > hostname instead of the IP address, and for 10.0.0.1, but 10.0.0.0 is
> > > > allowed)
> > > >
> > > > I am trying to get hbase running for a socorro server, which will
> > running
> > > > locally. I don't know if that matters.
> > > >
> > > > Thanks,
> > > > Michael
> > > >
> > > > On Wed, Sep 15, 2010 at 6:04 PM, Ryan Rawson <[email protected]>
> > wrote:
> > > >
> > > >> Hey,
> > > >>
> > > >> If you bind to localhost you wont actually be reachable by anyone!
> > > >>
> > > >> The question is why is your OS disallowing binds to a specific
> > > >> interface/port combo?
> > > >>
> > > >> HBase does not really run in a blended/multihomed environment...
> > > >> meaning if you have multiple interfaces, you have to choose one that
> > > >> we work over. This is because we need to know a singular canonical
> > > >> IP/name for any given server because we put that info up inside
> > > >> ZooKeeper and META tables. So it's not just an artificial
> constraint,
> > > >> but exists for cluster management needs.
> > > >>
> > > >> Having said that, we do work on multihomed machines, eg: ec2, you
> > > >> might bind hbase to the internal interface taking advantage of the
> > > >> unmetered/faster network. Also better for security as well.
> > > >>
> > > >> Let us know if you need more background on how we use the network
> and
> > > why.
> > > >> -ryan
> > > >>
> > > >> On Wed, Sep 15, 2010 at 10:18 AM, Michael Scott <
> [email protected]
> > >
> > > >> wrote:
> > > >> > Hi again,
> > > >> >
> > > >> > I think the hbase server master is not starting because it is
> > > attempting
> > > >> to
> > > >> > open port 60000 on its public IP address, rather than using
> > localhost.
> > > I
> > > >> > cannot seem to figure out how to force it (well, configure it) to
> > > attempt
> > > >> to
> > > >> > bind to localhost:60000 instead. As far as I can see, this is
> set
> > in
> > > >> the
> > > >> > file:
> > > >> >
> > > >> > org/apache/hadoop/hbase/master/HMaster.java
> > > >> >
> > > >> > I don't know much about java, so I'd prefer not to edit the source
> > if
> > > >> there
> > > >> > is an option, but I will if necessary. Can someone please point
> me
> > to
> > > >> the
> > > >> > way to change this setting? Any help would be greatly
> appreciated.
> > > >> >
> > > >> > Thanks,
> > > >> > Michael
> > > >> >
> > > >> > On Wed, Sep 15, 2010 at 12:42 AM, Michael Scott <
> > [email protected]
> > > >> >wrote:
> > > >> >
> > > >> >> Hi again,
> > > >> >>
> > > >> >> IPV6 was enabled. I shut it off, rebooted to be sure, verified
> it
> > > was
> > > >> >> still off, and encountered the same problem once again.
> > > >> >>
> > > >> >> I also tried to open port 60000 by hand with a small php file. I
> > can
> > > do
> > > >> >> this (as any user) for localhost. I can NOT do this (not even as
> > > root)
> > > >> for
> > > >> >> the IP address which matches the fully qualified domain name,
> which
> > > is
> > > >> what
> > > >> >> hbase is trying to use. Is there some way for me to configure
> > hbase
> > > to
> > > >> use
> > > >> >> localhost instead of the fully qualified domain name for the
> > master?
> > > I
> > > >> >> would have thought this was done by default, or that there would
> be
> > > an
> > > >> >> obvious line in some conf file, but I can't find it.
> > > >> >>
> > > >> >> Thanks again,
> > > >> >>
> > > >> >> Michael
> > > >> >>
> > > >> >> On Tue, Sep 14, 2010 at 12:23 PM, Todd Lipcon <[email protected]
> >
> > > >> wrote:
> > > >> >>
> > > >> >>> Hi Michael,
> > > >> >>>
> > > >> >>> It might be related to IPV6. Do you have IPV6 enabled on this
> > > machine?
> > > >> >>>
> > > >> >>> Check out this hadoop JIRA that might be related for some tips:
> > > >> >>> https://issues.apache.org/jira/browse/HADOOP-6056
> > > >> >>>
> > > >> >>> <https://issues.apache.org/jira/browse/HADOOP-6056>-Todd
> > > >> >>>
> > > >> >>> On Tue, Sep 14, 2010 at 10:17 AM, Michael Scott <
> > > [email protected]
> > > >> >>> >wrote:
> > > >> >>>
> > > >> >>> > That's correct. I tried a number of different ports to see if
> > > there
> > > >> was
> > > >> >>> > something weird, and then I shut down the hadoop server and
> > tried
> > > to
> > > >> >>> > connect
> > > >> >>> > to 50010 (which of course should have been free at that point)
> > but
> > > >> got
> > > >> >>> the
> > > >> >>> > same "cannot assign to requested address" error. If I start
> > > hadoop,
> > > >> >>> > netstat
> > > >> >>> > shows a process listening on 50010.
> > > >> >>> >
> > > >> >>> > I am going to try this on a different OS, I am wondering if
> FC11
> > > is
> > > >> my
> > > >> >>> > problem.
> > > >> >>> >
> > > >> >>> > Michael
> > > >> >>> >
> > > >> >>> > On Tue, Sep 14, 2010 at 11:41 AM, Stack <[email protected]>
> > wrote:
> > > >> >>> >
> > > >> >>> > > On Tue, Sep 14, 2010 at 9:33 AM, Michael Scott <
> > > >> [email protected]>
> > > >> >>> > > wrote:
> > > >> >>> > > > I don't see why hadoop binds
> > > >> >>> > > > to a port but hbase does not (I even tried starting hbase
> > with
> > > >> >>> hadoop
> > > >> >>> > off
> > > >> >>> > > > and binding to 50010, which hadoop uses).
> > > >> >>> > > >
> > > >> >>> > >
> > > >> >>> > > Using 50010 worked for hadoop but not for hbase? (Odd. We
> > > hadoop
> > > >> >>> > > their mechanism essentially).
> > > >> >>> > >
> > > >> >>> > > St.Ack
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >> >>>
> > > >> >>>
> > > >> >>> --
> > > >> >>> Todd Lipcon
> > > >> >>> Software Engineer, Cloudera
> > > >> >>>
> > > >> >>
> > > >> >>
> > > >> >
> > > >>
> > > >
> > >
> >
>
