Following is my understanding. Please correct me if there are any mistakes.

Did you change Zookeeper quorum configuration?
Zookeeper tracks table schema and regionservers. If ZK quorum changes,
the data Zookeeper saved is corrupted since the integration of the
data has been destroyed.
You may see some snippets from the log which says  the cluster can't
connect to a previously selected ZK node.

As far as I know, table data will not be lost after cleaning up the
zookeeper data, but I'm not sure.

On Fri, Jul 29, 2011 at 11:33 AM, Xian Woo <[email protected]> wrote:
> Wow,it really helps~~ Thanks pal~~
> But in fact, I really don't understand why this issue can be solved just by
> cleaning up the zookeeper data on each zk node.  So I really hope you can
> tell me something more about that. Thanks~
> Best wishes~
> Woo
> 2011/7/29 Gan, Xiyun <[email protected]>
>>
>> Hi, Woo
>>     I have been confronted with this issue in our cluster. Have you
>> started hbase cluster successfully ever before? If yes, try to clean
>> up the zookeeper data on each zk node.
>>
>>
>> On Fri, Jul 29, 2011 at 1:03 AM, Xian Woo <[email protected]> wrote:
>> > Excuse me for clicking the 'send' button accidently..
>> >
>> > Log in Server4.yun.com:
>> > Here are some segments:
>> >
>> > 2011-07-28 20:48:41,123 DEBUG
>> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=1.64
>> > MB,
>> > free=197.54 MB, max=199.18 MB, blocks=0, accesses=0, hits=0,
>> > hitRatio=�%,
>> > cachingAccesses=0, cachingHits=0, cachingHitsRatio=�%, evictions=0,
>> > evicted=0, evictedPerRun=NaN
>> >
>> > 2011-07-28 20:38:41,109 INFO org.apache.hadoop.ipc.HBaseServer: IPC
>> > Server
>> > handler 2 on 60020: starting
>> >
>> > 2011-07-28 20:38:40,598 DEBUG
>> > org.apache.hadoop.hbase.executor.ExecutorService: Starting executor
>> > service
>> > name=RS_OPEN_ROOT-server4.yun.com,60020,1311856719820, corePoolSize=1,
>> > maxPoolSize=1
>> >
>> > 2011-07-28 20:38:40,261 INFO
>> > org.apache.hadoop.hbase.regionserver.HRegionServer: Runs every
>> > 10000000ms
>> >
>> >
>> > Log in server5.yun.com:
>> >
>> > 1.It first keeps showing:
>> >
>> > 2011-07-28 20:37:05,616 DEBUG
>> > org.apache.hadoop.hbase.regionserver.HRegionServer:
>> > NotServingRegionException; Region is not online: -ROOT-,,0
>> >
>> > 2. Then it writes this per 5 minutes:
>> >
>> > 2011-07-28 20:40:52,814 DEBUG
>> > org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=1.64
>> > MB,
>> > free=197.54 MB, max=199.18 MB, blocks=0, accesses=0, hits=0,
>> > hitRatio=锟?,
>> > cachingAccesses=0, cachingHits=0, cachingHitsRatio=锟?, evictions=0,
>> > evicted=0, evictedPerRun=NaN
>> >
>> >
>> > I would really appreciate it if  you can give me some advice on this
>> > issue.
>> >
>> > Thanks a lot ~
>> >
>> > Best Wishes~
>> >
>> > Woo
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > 2011/7/29 Xian Woo <[email protected]>
>> >
>> >>
>> >> Hi everyone. I have just completed in installing my hbase and there are
>> >> some errors when I try to start my cluster.
>> >>
>> >> Setup:
>> >>    -cdh3u1
>> >>    - Hadoop 0.20.2
>> >>    - HBase 0.90.1
>> >>    - 1 Master Node running as NameNode & JobTracker(named
>> >> server3.yun.com)
>> >>    -zookeeper quorum
>> >>    - 2 child nodes(server4.yun.com & server5.yun.com) running as
>> >> Datanode,
>> >> TaskTracker and RegionServer each.
>> >>
>> >> When I try to start the master and the region servers, I find that some
>> >> issues in my log:
>> >>
>> >> Master log(server3.yun.com):
>> >>
>> >> 1. It first keeps showing this:
>> >> "2011-07-28 20:30:16,056 INFO
>> >> org.apache.hadoop.hbase.master.AssignmentManager: Region has been
>> >> OPENING
>> >> for too long, reassigning region=-ROOT-,,0.70236052
>> >> 2011-07-28 20:30:16,057 DEBUG
>> >> org.apache.hadoop.hbase.master.AssignmentManager: Region has
>> >> transitioned to
>> >> OPENED, allowing watched event handlers to process"
>> >>
>> >> 2.After several minutes of the same contents written in the log,  then:
>> >> "2011-07-28 20:30:16,412 DEBUG
>> >>
>> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
>> >> Lookedup root region location,
>> >>
>> >> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7e24d517;
>> >> hsa=server5.yun.com:60020
>> >> 2011-07-28 20:30:16,413 DEBUG
>> >>
>> >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
>> >> locateRegionInMeta parentTable=-ROOT-, metaLocation=address:
>> >> server5.yun.com:60020, regioninfo: -ROOT-,,0.70236052, attempt=1 of 10
>> >> failed; retrying after sleep of 1000 because:
>> >> org.apache.hadoop.hbase.NotServingRegionException: Region is not
>> >> online:
>> >> -ROOT-,,0
>> >>  at
>> >>
>> >> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2318)
>> >> at
>> >>
>> >> org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1614)
>> >>  at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>> >> at
>> >>
>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >>  at java.lang.reflect.Method.invoke(Method.java:597)
>> >> at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
>> >>  at
>> >>
>> >> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
>> >> "
>> >>
>> >> 3.Then it moves back to 1 and continues checking...
>> >>
>> >> Log
>> >>
>> >
>>
>>
>>
>> --
>> Best wishes
>> Gan, Xiyun
>
>



-- 
Best wishes
Gan, Xiyun

Reply via email to