Hi, I am on trunk and testing in pseudo distributed setup. I loaded the machine with YCSB and got it to break at a few million inserts during the load phase with the GC taking too long and the compaction queue going through the roof subsequently. Since then I cannot recover the local "cluster". It is stuck printing this:
... 2011-05-16 06:59:05,389 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired /hbase/splitlog/RESCAN0000148501 ver = 0 2011-05-16 06:59:06,388 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 1 unassigned = 1 2011-05-16 06:59:06,389 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: resubmitting unassigned task(s) after timeout 2011-05-16 06:59:06,390 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired /hbase/splitlog/RESCAN0000148502 ver = 0 2011-05-16 06:59:07,388 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 1 unassigned = 1 2011-05-16 06:59:07,388 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: resubmitting unassigned task(s) after timeout 2011-05-16 06:59:07,389 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired /hbase/splitlog/RESCAN0000148503 ver = 0 2011-05-16 06:59:08,388 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 1 unassigned = 1 2011-05-16 06:59:08,388 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: resubmitting unassigned task(s) after timeout 2011-05-16 06:59:08,389 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired /hbase/splitlog/RESCAN0000148504 ver = 0 2011-05-16 06:59:09,388 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 1 unassigned = 1 2011-05-16 06:59:09,389 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: resubmitting unassigned task(s) after timeout 2011-05-16 06:59:09,390 DEBUG org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired /hbase/splitlog/RESCAN0000148505 ver = 0 ... This keeps on going up and up. What is the right way to recover from this? Delete something from ZK? Delete something from HDFS? What shell commands would help? Thanks, Lars
