Have you checked master to see if region assignment went okay ? Cheers
> On Oct 12, 2015, at 7:56 AM, Jurian Broertjes <[email protected]> > wrote: > > Hi all, > > I'm using hbase (1.1.2) with phoenix (4.5.2-HBase-1.1) and had some (minor) > HDFS issues. The HDFS issues are resolved and when I try to bring HBase back > up, I run into issues where some regions won't come online. > > Some RS log: > 2015-10-12 14:08:54,681 INFO > [cn1.xxx.xx,16020,1444658908121-recovery-writer--pool6-t1] > client.AsyncProcess: #8, waiting for 1 actions to finish > 2015-10-12 14:08:55,781 INFO > [cn1.xxx.xx,16020,1444658908121-recovery-writer--pool6-t2] > client.AsyncProcess: #13, waiting for 1 actions to finish > 2015-10-12 14:08:55,819 INFO > [cn1.xxx.xx,16020,1444658908121-recovery-writer--pool6-t3] > client.AsyncProcess: #15, waiting for 2 actions to finish > 2015-10-12 14:08:59,119 INFO > [cn1.xxx.xx,16020,1444658908121-recovery-writer--pool6-t4] > client.AsyncProcess: #24, waiting for 1 actions to finish > 2015-10-12 14:08:59,138 INFO > [cn1.xxx.xx,16020,1444658908121-recovery-writer--pool6-t5] > client.AsyncProcess: #25, waiting for 2 actions to finish > 2015-10-12 14:09:04,692 INFO > [cn1.xxx.xx,16020,1444658908121-recovery-writer--pool6-t1] > client.AsyncProcess: #8, waiting for 1 actions to finish > 2015-10-12 14:09:05,793 INFO > [cn1.xxx.xx,16020,1444658908121-recovery-writer--pool6-t2] > client.AsyncProcess: #13, waiting for 1 actions to finish > 2015-10-12 14:09:05,831 INFO > [cn1.xxx.xx,16020,1444658908121-recovery-writer--pool6-t3] > client.AsyncProcess: #15, waiting for 2 actions to finish > 2015-10-12 14:09:07,214 INFO > [regionserver/cn1.xxx.xx/89.188.14.2:16020-shortCompactions-1444658915963] > client.AsyncProcess: #23, waiting for some tasks to finish. Expected max=0, > tasksInProgress=9 > 2015-10-12 14:09:09,131 INFO > [cn1.xxx.xx,16020,1444658908121-recovery-writer--pool6-t4] > client.AsyncProcess: #24, waiting for 1 actions to finish > 2015-10-12 14:09:09,150 INFO > [cn1.xxx.xx,16020,1444658908121-recovery-writer--pool6-t5] > client.AsyncProcess: #25, waiting for 2 actions to finish > 2015-10-12 14:09:12,945 INFO [htable-pool8-t1] client.AsyncProcess: #8, > table=OUTLINKS_SSI_INDEX, attempt=10/350 failed=1ops, last exception: > org.apache.hadoop.hbase.NotServingRegionException: > org.apache.hadoop.hbase.NotServingRegionException: Region > OUTLINKS_SSI_INDEX,,1440761791894.c9cfcf16be9852553efe45e36387a4b1. is not > online on cn1.xxx.xx,16020,1444658908121 > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2898) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:947) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1991) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101) > at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) > at java.lang.Thread.run(Thread.java:745) > on cn1.xxx.xx,16020,1444654232108, tracking started null, retrying > after=10086ms, replay=1ops > > The cluster consists of 2 masters and 3 region servers and an external > Zookeeper. > > Anyone knows what's going on here? > > Thanks in advance, > > Jurian
