----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1078/#review1655 -----------------------------------------------------------
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionSelectorHBase89.java <http://review.cloudera.org/r/1078/#comment5531> not sure if you saw it but https://issues.apache.org/jira/browse/HBASE-3035 is in 0.89 (but not trunk) in order to take care of that. - Jean-Daniel On 2010-10-23 00:48:28, Jonathan Gray wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > http://review.cloudera.org/r/1078/ > ----------------------------------------------------------- > > (Updated 2010-10-23 00:48:28) > > > Review request for hbase, stack, Nicolas, Karthik Ranganathan, and Kannan > Muthukkaruppan. > > > Summary > ------- > > Pulls compaction file selection code into new interface and makes it so it's > configurable. Currently is globally configurable but should be easy to make > it per-family setting. > > Also makes the algorithm standalone and testable. > > Includes a new compaction algorithm based on a new config param > 'compactionForce'. See javadoc in compaction classes for explanation. > > Big test included for new algorithm. > > Also the TestCompact class includes a neat new way for us to compare > compaction algorithms. You specify a bunch of input paramaters and then it > runs a simulation and generates statistics. The output looks like: > > > ----- > Ran test > ----- > numPuts=1000000 > putSizeRange=1.0KB to 10.0KB > numPutsPerGet=10 > flushSizeRange=64.0MB to 256.0MB > max=10, threshold=3, force=6, factor=0.5 > ----- > > ----- > Final Result > ----- > files=82.2MB, 2.9GB, 898.3MB, 1.3GB > memstoreSize=100.8MB > totalSize=5.1GB > totalThroughput=18.2GB > averageFilesPerGet=3.25622 > > > This addresses bug HBASE-2462. > http://issues.apache.org/jira/browse/HBASE-2462 > > > Diffs > ----- > > trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java 1026565 > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java > 1026565 > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > 1026565 > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > 1026565 > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java > 1026565 > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/PriorityCompactionQueue.java > 1026565 > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java > 1026565 > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > 1026565 > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1026565 > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactSplitThread.java > PRE-CREATION > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionRequestor.java > PRE-CREATION > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionSelector.java > PRE-CREATION > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionSelectorHBase89.java > PRE-CREATION > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionSelectorWithForce.java > PRE-CREATION > > trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java > 1026565 > > trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestPriorityCompactionQueue.java > 1026565 > > trunk/src/test/java/org/apache/hadoop/hbase/regionserver/compact/TestCompact.java > PRE-CREATION > > Diff: http://review.cloudera.org/r/1078/diff > > > Testing > ------- > > TestCompact is passing. Have not run test suite. > > > Thanks, > > Jonathan > >
