bq. Found 1 replicas but expecting no less than 3 replicas. This is controlled by "hbase.regionserver.hlog.tolerable.lowreplication" You can lower the value of the above config.
Cheers On Thu, Oct 30, 2014 at 10:16 PM, Li Li <[email protected]> wrote: > hi all, > I am using hbase and also phoenix(some tables are managed by myself > and some are created by phoenix). > Last night, the disk is full . I killed the hbase > and hadoop related processes. But After that I can't start hbase > anymore. > I am using ubuntu 12.04 and hadoop-1.2.1 and hbase 0.98.5 with phoenix > 4.1.0 > The region server print error like: > > 2014-10-31 12:33:49,835 INFO > [mobvoi-knowledge-graph-0,60020,1414729587199-recovery-writer--pool4-t2] > client.AsyncProcess: #9, waiting for some tasks to finish. Expected > max=0, tasksSent=31, tasksDone=30, currentTasksDone=30, retries=30 > hasError=false, tableName=BAIDUMUSIC.BAIDUMUSIC_IDX > > 2014-10-31 12:33:49,842 INFO [htable-pool9-t1] client.AsyncProcess: > #9, table=BAIDUMUSIC.BAIDUMUSIC_IDX, attempt=31/350 failed 1 ops, last > exception: org.apache.hadoop.hbase.exceptions.RegionOpeningException: > org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region > > BAIDUMUSIC.BAIDUMUSIC_IDX,\x0D\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1414488087936.69e9a7efb9ee00b1ecfe50f825e7cc5b. > is opening on mobvoi-knowledge-graph-0,60020,1414729587199 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2692) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4139) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3363) > > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29593) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2026) > > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98) > > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114) > > at > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) > > at java.lang.Thread.run(Thread.java:745) > > on mobvoi-knowledge-graph-0,60020,1414484323072, tracking started Fri > Oct 31 12:26:39 CST 2014, retrying after 20119 ms, replay 1 ops. > > 2014-10-31 12:33:55,747 INFO > [mobvoi-knowledge-graph-0,60020,1414729587199-recovery-writer--pool4-t3] > client.AsyncProcess: #25, waiting for some tasks to finish. Expected > max=0, tasksSent=31, tasksDone=30, currentTasksDone=30, retries=30 > hasError=false, tableName=BAIDUMUSIC.BAIDUMUSIC_IDX > > 2014-10-31 12:33:55,755 INFO [htable-pool24-t1] client.AsyncProcess: > #25, table=BAIDUMUSIC.BAIDUMUSIC_IDX, attempt=31/350 failed 2 ops, > last exception: > org.apache.hadoop.hbase.exceptions.RegionOpeningException: > org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region > > BAIDUMUSIC.BAIDUMUSIC_IDX,\x0C\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1414488087936.378bef2d741a9c00761160670329fc7c. > is opening on mobvoi-knowledge-graph-0,60020,1414729587199 > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2692) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4139) > > at > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3363) > > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29593) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2026) > > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98) > > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114) > > at > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) > > at java.lang.Thread.run(Thread.java:745) > > on mobvoi-knowledge-graph-0,60020,1414484323072, tracking started Fri > Oct 31 12:26:45 CST 2014, retrying after 20001 ms, replay 2 ops. > > I also find warns like > wal.FSHLog: HDFS pipeline error detected. Found 1 replicas but > expecting no less than 3 replicas. Requesting close of hlog. > But my hbase cluster is pesudo cluster that all processes are running > in a single machine and the hdfs (I have set dfs.replication=1 in > hdfs-site.xml) I have tried to set replication factor to 1 by hadoop > fs -setrep -R 1 / but no luck >
