If MemStore flush fails RS aborts. As per the below stack trace I find that the flushing was not Successful due to some failure in HDFS. Can you check the NN and DN logs at this time?
Regards Ram -----Original Message----- From: xiaochao [mailto:[email protected]] Sent: Saturday, December 31, 2011 9:02 AM To: [email protected] Subject: hbase RegionServer dead suddenly Hello: hbase RegionServer is dead with the log below.can someone help me? 2011-12-28 12:12:34,899 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server serverName=192.168.3.39,50012,1324979072340, load= (requests=403, regions=1398, usedHeap=3731, maxHeap=9983): Replay of HLog required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: guard_tb_20111226,Szzzzzzzzzzz,1322710771589.89644afd8e35f06181706e87 4747129e. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HReg ion.java:995) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HReg ion.java:900) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java :852) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(Mem StoreFlusher.java:392) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlob alPressure(MemStoreFlusher.java:200) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlu sher.java:220) Caused by: java.io.IOException: Bad connect ack with firstBadLink as 192.168.3.37:50001 at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.createBlockOutputSt ream(DFSClient.java:3139) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStre am(DFSClient.java:3055) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1900(DFSClie nt.java:2305) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DF SClient.java:2500)
