I know I didn't have a clean shutdown, I thought I had hit HBASE-3038, but looking further I first had a OOME on a region server. Can I revert to the oder HBASE to reconstruct the log or has that ship sailed?
thanks, -chris On Mar 17, 2011, at 4:22 PM, Ryan Rawson wrote: > If you know you had a clean shutdown just nuke all directories in /hbase/.logs > > we hit this @ SU as well, its older logfile formats messing us up. > > remember, only if you had a CLEAN shutdown, or else you lose data!!!! > > On Thu, Mar 17, 2011 at 4:20 PM, Chris Tarnas <[email protected]> wrote: >> >> >> I just had to upgrade our second cluster CDH3B4 (the 2GB log file problem, >> same as the reason for upgrading another cluster) and now the master is not >> coming up, it dies with this error: >> >> >> 2011-03-17 18:15:24,209 FATAL org.apache.hadoop.hbase.master.HMaster: >> Unhandled exception. Starting shutdown. >> java.lang.RuntimeException: java.lang.IllegalArgumentException: >> java.net.URISyntaxException: Relative path in absolute URI: >> sequence,lists-Gbaa-KOdBQHTxUyTq8MAwGA10:4:16:629:647%230/1Nr24og9ZJoEEzRue1qKSCg%09GA10:4:16:629:647%230/1%09,1300314038804.2e7bdb018c92a7e22be79f21fcb6bee6. >> at >> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.checkForErrors(HLogSplitter.java:461) >> at >> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.access$100(HLogSplitter.java:66) >> at >> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$OutputSink.finishWritingAndClose(HLogSplitter.java:745) >> at >> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLog(HLogSplitter.java:300) >> at >> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLog(HLogSplitter.java:188) >> at >> org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:196) >> at >> org.apache.hadoop.hbase.master.MasterFileSystem.splitLogAfterStartup(MasterFileSystem.java:180) >> at >> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:379) >> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:278) >> >> >> HDFS is fine.. fsck ran clean. >> >> Here is more of the master log: >> >> http://pastebin.com/Uq5Riczz >> >> Thanks for any help! >> -chris
