Hi. You need to figure out why your region servers are all reporting with the loopback address (127.0.0.1).
I encountered this problem before and in my case the cause was bogus entries in /etc/hosts: The OS installer (I use RedHat Linux) would write an entry into /etc/hosts that maps the configured host name to 127.0.0.1. Why the RedHat OS installer (and perhaps others) might do this is a debatable point. In my opinion, it should not. If something similar to this has happened in your environment, all of your region servers -- regardless of whatever IP addresses might be assigned to the Ethernet network interfaces and whatever hostname<->IP address mappings you may have set up in your local DNS zone -- will be reporting to the master with the same apparent identity (127.0.0.1). So definitely you should look at your networking configuration and you should start with /etc/hosts. Hope this helps, - Andy --- Jason Sheldon <[EMAIL PROTECTED]> wrote: > Hi > > Thanks for the feedback. I got a little further but now it tells me:- > > 2008-05-12 09:02:43,889 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 1 on 60000, call regionServerStartup(address: 127.0.1.1:60020, > > startcode: 1210352247220, load: (requests: 0 regions: 0)) from > 10.211.55.6:51202: error: org.apache.hadoop.hbase.Leases > $LeaseStillHeldException > org.apache.hadoop.hbase.Leases$LeaseStillHeldException > at org.apache.hadoop.hbase.Leases.createLease(Leases.java:147) > at org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: > 1311) > at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) > at > sun > .reflect > .DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: > > 25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413) > > I have wiped the /hbase directory and restarted and it does not seem > to help > > On 9 May 2008, at 18:12, stack wrote: > > > Hey Jason. Thanks for trying hbase. > > > > Can you take a look at your networking? Remote regionservers are > > reporting in as 127.0.1.1:60020 rather than as 10.211.55.X. > > > > The master keeps an in-memory Map of all regionservers. The key is > > the regionserver's reported Address+Port. If all are reporting in > > as 127.0.1.1:60020, then the master is confused (AssertionError). > > > > Yours, > > St.Ack > > > > P.S. Would suggest you also use the latest release candidate: > http://people.apache.org/~stack/hbase-0.1.2-candidate-1/ > > . It is much improved over 0.1.1. > > > > > > Jason Sheldon wrote: > >> Hi > >> > >> I am very new to hbase and have just set it up for the first time. > >> I have a running hadoop cluster with 5 datanodes and have installed > > >> HRegionServers on each of these nodes. > >> > >> When I start up hbase I see this error in the master log:- > >> > >> 2008-05-09 15:43:35,922 INFO org.apache.hadoop.ipc.Server: IPC > >> Server handler 7 on 60000, call regionServerStartup(address: > >> 127.0.1.1:60020, startcode: 1210304455421, load: (requests: 0 > >> regions: 0)) from 10.211.55.4:55809: error: java.io.IOException: > >> java.lang.AssertionError: Impossible state for createLease(): Lease > > >> 1377822776/1377822776 is still held. > >> java.io.IOException: java.lang.AssertionError: Impossible state for > > >> createLease(): Lease 1377822776/1377822776 is still held. > >> at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145) > >> at > >> org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: > >> 1310) > >> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) > >> at > >> sun > >> .reflect > >> .DelegatingMethodAccessorImpl > >> .invoke(DelegatingMethodAccessorImpl.java:25) > >> at java.lang.reflect.Method.invoke(Method.java:585) > >> at org.apache.hadoop.hbase.ipc.HbaseRPC > >> $Server.call(HbaseRPC.java:413) > >> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899) > >> > >> > >> in the regionservers I see > >> > >> 2008-05-09 15:45:05,655 WARN org.apache.hadoop.hbase.HRegionServer: > > >> error telling master we are up > >> org.apache.hadoop.ipc.RemoteException: java.io.IOException: > >> java.lang.AssertionError: Impossible state for createLease(): Lease > > >> 1377822776/1377822776 is still held. > >> at org.apache.hadoop.hbase.Leases.createLease(Leases.java:145) > >> at > >> org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java: > >> 1310) > >> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) > >> at > >> sun > >> .reflect > >> .DelegatingMethodAccessorImpl > >> .invoke(DelegatingMethodAccessorImpl.java:25) > >> at java.lang.reflect.Method.invoke(Method.java:585) > >> at org.apache.hadoop.hbase.ipc.HbaseRPC > >> $Server.call(HbaseRPC.java:413) > >> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899) > >> > >> at org.apache.hadoop.ipc.Client.call(Client.java:512) > >> at org.apache.hadoop.hbase.ipc.HbaseRPC > >> $Invoker.invoke(HbaseRPC.java:210) > >> at $Proxy0.regionServerStartup(Unknown Source) > >> at > >> org > >> .apache.hadoop.hbase.HRegionServer.reportForDuty(HRegionServer.java: > > >> 1037) > >> at org.apache.hadoop.hbase.HRegionServer.run(HRegionServer.java: > >> 634) > >> at java.lang.Thread.run(Thread.java:595) > >> > >> > >> As such I only see the master (localhost) region server as being > >> active > >> > >> HBase Version 0.1.1, r HBase version and svn revision > >> Hadoop Version 0.16.2, r642481 Hadoop version and svn revision > >> Filesystem hdfs://10.211.55.7:9000/hbase Filesystem HBase is > >> running on > >> HBase Root Directory hdfs://10.211.55.7:9000/hbase Location > >> of HBase home directory > >> Online META Regions > >> Name Server > >> -ROOT- 127.0.1.1:60020 > >> .META.,,1 127.0.1.1:60020 > >> Tables > >> > >> No tables found. > >> Region Servers > >> Address Start Code Load > >> 127.0.1.1:60020 1210340213164 requests: 0 regions: 2 > >> Total: servers: 1 requests: 0 regions: 2 > >> > >> > >> > >> Any explanation/help would be greatly appreciated > >> > >> Thanks > > > > Jason Sheldon > Platform Engineer > SynthaSite > > Web: http://www.synthasite.com > > > > ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
