----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/396/#review513 -----------------------------------------------------------
This looks good, except that it requires the HDFS patches to be in place to even compile. Although I think it's OK to not fix the bug in the case that we're on stock HDFS, I think we still need to be able to run, buggily. Did we determine that all of the other solutions were flawed/too complicated? trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java <http://review.cloudera.org/r/396/#comment2061> Check return value of rename (this is one of those stupid APIs that returns false instead of throwing) trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java <http://review.cloudera.org/r/396/#comment2062> rather than reassigning, just pass splitDir below? trunk/src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java <http://review.cloudera.org/r/396/#comment2064> extract "-splitting" out to a constant? Or can we move some of this common code into HLog? trunk/src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java <http://review.cloudera.org/r/396/#comment2063> maybe: assert !(logDirExists && splitDirExists) : "Both " + rsLogDir + " and " + rsSplitDir + " exist"; so if the assertion failure happens it's more understandable - Todd On 2010-07-27 23:10:02, Nicolas wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > http://review.cloudera.org/r/396/ > ----------------------------------------------------------- > > (Updated 2010-07-27 23:10:02) > > > Review request for hbase. > > > Summary > ------- > > There is a very corner case when bad things could happen(ie data loss): > > 1) RS #1 is going to roll its HLog - not yet created the new one, old one > will get no more writes > 2) RS #1 enters GC Pause of Death > 3) Master lists HLog files of RS#1 that is has to split as RS#1 is dead, > starts splitting > 4) RS #1 wakes up, created the new HLog (previous one was rolled) and appends > an edit - which is lost > > Note that this fix requires a healthy dose of HDFS prerequisites: HDFS-617, > HADOOP-6840, HADOOP-6886. I encourage you to review those as well, give > feedback, and hopefully give +1s so we can push the changes through. > > > This addresses bug HBASE-2312. > http://issues.apache.org/jira/browse/HBASE-2312 > > > Diffs > ----- > > trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 979953 > > trunk/src/main/java/org/apache/hadoop/hbase/master/ProcessServerShutdown.java > 979953 > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java > 979953 > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogWriter.java > 979953 > > trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java > 979953 > > Diff: http://review.cloudera.org/r/396/diff > > > Testing > ------- > > mvn test; > bin/start-hbase.sh > bin/hbase shell < scan '.META.', get, put, etc > > > Thanks, > > Nicolas > >
