Check if there are an 0 sized wals in /hbase/.logs and sideline them and restart. That could help. As Ted mentioned the actual problematic log names are in the RS logs that got the task assigned.
On Fri, Mar 7, 2014 at 12:43 AM, David Koch <[email protected]> wrote: > Hello, > > Our HBase cluster had an unexpected shut-down and while trying to bring it > back up we the Master gets stuck with the following message: > > Failed splitting of [ list of <host_name>,<port>,<tmst> ] > java.io.IOException: error or interrupted while splitting logs in [ list of > <host_name>,<port>,<tmst> ] > Task = installed = 10 done = 0 error = 10 > at > > org.apache.hadoop.hbase.master.SplitLogManager.splitLogDistributed(SplitLogManager.java:282) > at > > org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:300) > at > > org.apache.hadoop.hbase.master.MasterFileSystem.splitLogAfterStartup(MasterFileSystem.java:242) > at > > org.apache.hadoop.hbase.master.HMaster.splitLogAfterStartup(HMaster.java:661) > at > > org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:580) > at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:396) > at java.lang.Thread.run(Thread.java:724) > > What can I do to get the cluster operational again. There was no data > ingestion going on since quite some hours before the crash so maybe > clearing out /hbase/.logs/ could be an option. > > Thanks, > > /David > -- Bharath Vissapragada <http://www.cloudera.com>
