----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1078/#review1634 -----------------------------------------------------------
This is great. Don't we already have a standalone compaction issues? Should kill it w/ this issue? Some broad questions below. Do you find your new selector better than old? trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionSelector.java <http://review.cloudera.org/r/1078/#comment5493> What is an index range? Presumption is that we compact adjacent files? Might want to say that in javadoc? trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionSelectorHBase89.java <http://review.cloudera.org/r/1078/#comment5494> compaction stuff deserves its own subpackage now? trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java <http://review.cloudera.org/r/1078/#comment5495> Would be 'nicer' if selector returned List of StoreFiles rather than indices? trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java <http://review.cloudera.org/r/1078/#comment5496> I suppose compaction is single-threaded. Later when multi-threaded will have to do better here. trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java <http://review.cloudera.org/r/1078/#comment5497> Fall back on a default? - stack On 2010-10-22 23:39:07, Jonathan Gray wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > http://review.cloudera.org/r/1078/ > ----------------------------------------------------------- > > (Updated 2010-10-22 23:39:07) > > > Review request for hbase, stack, Nicolas, Karthik Ranganathan, and Kannan > Muthukkaruppan. > > > Summary > ------- > > Pulls compaction file selection code into new interface and makes it so it's > configurable. Currently is globally configurable but should be easy to make > it per-family setting. > > Also makes the algorithm standalone and testable. > > Includes a new compaction algorithm based on a new config param > 'compactionForce'. See javadoc in compaction classes for explanation. > > Big test included for new algorithm. > > Also the TestCompact class includes a neat new way for us to compare > compaction algorithms. You specify a bunch of input paramaters and then it > runs a simulation and generates statistics. The output looks like: > > > ----- > Ran test > ----- > numPuts=1000000 > putSizeRange=1.0KB to 10.0KB > numPutsPerGet=10 > flushSizeRange=64.0MB to 256.0MB > max=10, threshold=3, force=6, factor=0.5 > ----- > > ----- > Final Result > ----- > files=82.2MB, 2.9GB, 898.3MB, 1.3GB > memstoreSize=100.8MB > totalSize=5.1GB > totalThroughput=18.2GB > averageFilesPerGet=3.25622 > > > This addresses bug HBASE-2462. > http://issues.apache.org/jira/browse/HBASE-2462 > > > Diffs > ----- > > trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java 1026565 > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionSelector.java > PRE-CREATION > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionSelectorHBase89.java > PRE-CREATION > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionSelectorWithForce.java > PRE-CREATION > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1026565 > trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompact.java > PRE-CREATION > > Diff: http://review.cloudera.org/r/1078/diff > > > Testing > ------- > > TestCompact is passing. Have not run test suite. > > > Thanks, > > Jonathan > >
