[
https://issues.apache.org/jira/browse/HBASE-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924147#action_12924147
]
HBase Review Board commented on HBASE-2462:
-------------------------------------------
Message from: "Jonathan Gray" <[email protected]>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1078/
-----------------------------------------------------------
(Updated 2010-10-23 00:48:28.858137)
Review request for hbase, stack, Nicolas, Karthik Ranganathan, and Kannan
Muthukkaruppan.
Changes
-------
Working patch.
Summary
-------
Pulls compaction file selection code into new interface and makes it so it's
configurable. Currently is globally configurable but should be easy to make it
per-family setting.
Also makes the algorithm standalone and testable.
Includes a new compaction algorithm based on a new config param
'compactionForce'. See javadoc in compaction classes for explanation.
Big test included for new algorithm.
Also the TestCompact class includes a neat new way for us to compare compaction
algorithms. You specify a bunch of input paramaters and then it runs a
simulation and generates statistics. The output looks like:
-----
Ran test
-----
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=6, factor=0.5
-----
-----
Final Result
-----
files=82.2MB, 2.9GB, 898.3MB, 1.3GB
memstoreSize=100.8MB
totalSize=5.1GB
totalThroughput=18.2GB
averageFilesPerGet=3.25622
This addresses bug HBASE-2462.
http://issues.apache.org/jira/browse/HBASE-2462
Diffs (updated)
-----
trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java 1026565
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
1026565
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1026565
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
1026565
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
1026565
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/PriorityCompactionQueue.java
1026565
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
1026565
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
1026565
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1026565
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactSplitThread.java
PRE-CREATION
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionRequestor.java
PRE-CREATION
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionSelector.java
PRE-CREATION
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionSelectorHBase89.java
PRE-CREATION
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compact/CompactionSelectorWithForce.java
PRE-CREATION
trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
1026565
trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestPriorityCompactionQueue.java
1026565
trunk/src/test/java/org/apache/hadoop/hbase/regionserver/compact/TestCompact.java
PRE-CREATION
Diff: http://review.cloudera.org/r/1078/diff
Testing
-------
TestCompact is passing. Have not run test suite.
Thanks,
Jonathan
> Review compaction heuristic and move compaction code out so standalone and
> independently testable
> -------------------------------------------------------------------------------------------------
>
> Key: HBASE-2462
> URL: https://issues.apache.org/jira/browse/HBASE-2462
> Project: HBase
> Issue Type: Improvement
> Reporter: stack
> Assignee: Jonathan Gray
> Priority: Critical
>
> Anything that improves our i/o profile makes hbase run smoother. Over in
> HBASE-2457, good work has been done already describing the tension between
> minimizing compactions versus minimizing count of store files. This issue is
> about following on from what has been done in 2457 but also, breaking the
> hard-to-read compaction code out of Store.java out to a standalone class that
> can be the easier tested (and easily analyzed for its performance
> characteristics).
> If possible, in the refactor, we'd allow specification of alternate merge
> sort implementations.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
