Thank you very much, J-D. I was trapped by the problem for a long time.
Thank you again. I will upgrade to 0.20.6. Best regards, Xiujin Yang. > Date: Wed, 25 Aug 2010 09:30:55 -0700 > Subject: Re: Region servers down... > From: [email protected] > To: [email protected] > > That's https://issues.apache.org/jira/browse/HBASE-2797, please > upgrade to 0.20.6 (no migration needed, just copy over the configs). > > J-D > > 2010/8/24 xiujin yang <[email protected]>: > > > > Thank you J-D. > > > > The out file is like this. It has an "NullPointerException" error. > > > > 2010-08-24 02:30:14.187::INFO: Logging to STDERR via > > org.mortbay.log.StdErrLog > > 2010-08-24 02:30:14.187::INFO: jetty-6.1.14 > > 2010-08-24 02:30:14.122::INFO: Started [email protected]:60030 > > Exception in thread "regionserver/192.168.158.187:60020.leaseChecker" > > java.lang.NullPointerException > > at > > org.apache.hadoop.hbase.regionserver.ReadWriteConsistencyControl.getThreadReadPoint(ReadWriteConsistencyControl.java:40) > > at > > org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.getNext(MemStore.java:532) > > at > > org.apache.hadoop.hbase.regionserver.MemStore$MemStoreScanner.seek(MemStore.java:558) > > at > > org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:320) > > at > > org.apache.hadoop.hbase.regionserver.StoreScanner.checkReseek(StoreScanner.java:306) > > at > > org.apache.hadoop.hbase.regionserver.StoreScanner.peek(StoreScanner.java:143) > > at > > org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:127) > > at > > org.apache.hadoop.hbase.regionserver.KeyValueHeap$KVScannerComparator.compare(KeyValueHeap.java:117) > > at > > java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:641) > > at java.util.PriorityQueue.siftDown(PriorityQueue.java:612) > > at java.util.PriorityQueue.poll(PriorityQueue.java:523) > > at > > org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:151) > > at > > org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.close(HRegion.java:1971) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer$ScannerListener.leaseExpired(HRegionServer.java:1962) > > at org.apache.hadoop.hbase.Leases.run(Leases.java:98) > > > > > >> Date: Tue, 24 Aug 2010 11:16:34 -0700 > >> Subject: Re: Region servers down... > >> From: [email protected] > >> To: [email protected] > >> > >> The last log to look at would be the .out file. > >> > >> J-D > >> > >> 2010/8/23 xiujin yang <[email protected]>: > >> > > >> > Thank you J-D, > >> > > >> > I posted today's whole RS log: > >> > http://pastebin.com/djGnNJxk > >> > > >> > GC log: > >> > http://pastebin.com/AQH5kUCE > >> > > >> > I don't see the messages started with "We slept". > >> > > >> > > >> > > >> > > >> >> Date: Mon, 23 Aug 2010 23:00:32 -0700 > >> >> Subject: Re: Region servers down... > >> >> From: [email protected] > >> >> To: [email protected] > >> >> > >> >> I don't really see the cause of the shutdown in there, it seems it was > >> >> already under way. Do you see messages starting with "We slept" and > >> >> then telling how long it slept? It should be not very far from that in > >> >> the log. > >> >> > >> >> J-D > >> >> > >> >> 2010/8/23 xiujin yang <[email protected]>: > >> >> > > >> >> > Hi, > >> >> > > >> >> > RS of HBase was frequently down when running. And job will failed > >> >> > after the region server down. > >> >> > > >> >> > [regionserver/192.168.158.187:60020] 2010-08-24 04:15:14,929 INFO > >> >> > org.apache.hadoop.hbase.regionserver.HRegion: Closed > >> >> > whitetable,com.cnet.download:http/Seal-Online/3640-7540_4-10816413.html,1282619615378 > >> >> > [regionserver/192.168.158.187:60020] 2010-08-24 04:15:14,929 INFO > >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: telling master > >> >> > that region server is shutting down at: 192.168.158.187:60020 > >> >> > [regionserver/192.168.158.187:60020] 2010-08-24 04:15:14,929 INFO > >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: stopping server > >> >> > at: 192.168.158.187:60020 > >> >> > [regionserver/192.168.158.187:60020.worker] 2010-08-24 04:15:15,803 > >> >> > INFO org.apache.hadoop.hbase.regionserver.HRegionServer: worker > >> >> > thread exiting > >> >> > [regionserver/192.168.158.187:60020] 2010-08-24 04:15:15,829 INFO > >> >> > org.apache.zookeeper.ZooKeeper: Session: 0x12a9f2d85010005 closed > >> >> > [regionserver/192.168.158.187:60020] 2010-08-24 04:15:15,928 INFO > >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: > >> >> > regionserver/192.168.158.187:60020 exiting > >> >> > [Thread-17] 2010-08-24 04:15:16,115 INFO > >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: Starting shutdown > >> >> > thread. > >> >> > [Thread-17] 2010-08-24 04:15:16,115 INFO > >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: Shutdown thread > >> >> > complete > >> >> > [HCM.shutdownHook] 2010-08-24 04:15:16,115 INFO > >> >> > org.apache.zookeeper.ZooKeeper: Session: 0x12a9f2d85010041 closed > >> >> > > >> >> > > >> >> > Could anyone help me? > >> >> > > >> >> > Here is snippet from the region server log: > >> >> > http://pastebin.com/YCUDLqc3 > >> >> > > >> >> > Version: > >> >> > HBase: 0.20.5 > >> >> > Hadoop: 0.20.2 > >> >> > Zookeeper: 3.3.0 > >> >> > > >> >> > > >> >> > > >> > > >
