[
https://issues.apache.org/jira/browse/HBASE-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924142#action_12924142
]
Jonathan Gray commented on HBASE-2462:
--------------------------------------
Simulations of the new algorithm in the current patch up on RB:
(You can see a very clear trade-off between optimizing for writes or reads)
FINAL COMPARISON
2010-10-23 00:08:47,883 DEBUG [main] regionserver.TestCompact(188):
MOST EFFICIENT THROUGHPUT
2010-10-23 00:08:47,884 DEBUG [main] regionserver.TestCompact(190):
--- Config ---
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=6, factor=0.5
--- Result ---
files=null
memstoreSize=0B
totalSize=0B
totalThroughput=22.9GB
averageFilesPerGet=3.54831
2010-10-23 00:08:47,884 DEBUG [main] regionserver.TestCompact(190):
--- Config ---
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=3, factor=0.5
--- Result ---
files=null
memstoreSize=0B
totalSize=0B
totalThroughput=45.9GB
averageFilesPerGet=1.47172
2010-10-23 00:08:47,884 DEBUG [main] regionserver.TestCompact(190):
--- Config ---
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=3, factor=0.5
--- Result ---
files=null
memstoreSize=0B
totalSize=0B
totalThroughput=47.5GB
averageFilesPerGet=1.44104
2010-10-23 00:08:47,884 DEBUG [main] regionserver.TestCompact(190):
--- Config ---
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=3, factor=0.5
--- Result ---
files=null
memstoreSize=0B
totalSize=0B
totalThroughput=48.9GB
averageFilesPerGet=1.37419
2010-10-23 00:08:47,884 DEBUG [main] regionserver.TestCompact(190):
--- Config ---
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=3, factor=0.5
--- Result ---
files=null
memstoreSize=0B
totalSize=0B
totalThroughput=51.9GB
averageFilesPerGet=1.42109
2010-10-23 00:08:47,884 DEBUG [main] regionserver.TestCompact(194):
MOST EFFICIENT GETS
2010-10-23 00:08:47,885 DEBUG [main] regionserver.TestCompact(196):
--- Config ---
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=3, factor=0.5
--- Result ---
files=null
memstoreSize=0B
totalSize=0B
totalThroughput=48.9GB
averageFilesPerGet=1.37419
2010-10-23 00:08:47,885 DEBUG [main] regionserver.TestCompact(196):
--- Config ---
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=3, factor=0.5
--- Result ---
files=null
memstoreSize=0B
totalSize=0B
totalThroughput=51.9GB
averageFilesPerGet=1.42109
2010-10-23 00:08:47,885 DEBUG [main] regionserver.TestCompact(196):
--- Config ---
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=3, factor=0.5
--- Result ---
files=null
memstoreSize=0B
totalSize=0B
totalThroughput=47.5GB
averageFilesPerGet=1.44104
2010-10-23 00:08:47,885 DEBUG [main] regionserver.TestCompact(196):
--- Config ---
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=3, factor=0.5
--- Result ---
files=null
memstoreSize=0B
totalSize=0B
totalThroughput=45.9GB
averageFilesPerGet=1.47172
2010-10-23 00:08:47,886 DEBUG [main] regionserver.TestCompact(196):
--- Config ---
numPuts=1000000
putSizeRange=1.0KB to 10.0KB
numPutsPerGet=10
flushSizeRange=64.0MB to 256.0MB
max=10, threshold=3, force=6, factor=0.5
--- Result ---
files=null
memstoreSize=0B
totalSize=0B
totalThroughput=22.9GB
averageFilesPerGet=3.54831
2010-10-23 00:08:47,886 DEBUG [main] regionserver.TestCompact(200):
Throughput Range: 22.9GB to 51.9GB
2010-10-23 00:08:47,886 DEBUG [main] regionserver.TestCompact(203):
Num Files Per Get Range: 1.37419 to 3.54831
> Review compaction heuristic and move compaction code out so standalone and
> independently testable
> -------------------------------------------------------------------------------------------------
>
> Key: HBASE-2462
> URL: https://issues.apache.org/jira/browse/HBASE-2462
> Project: HBase
> Issue Type: Improvement
> Reporter: stack
> Assignee: Jonathan Gray
> Priority: Critical
>
> Anything that improves our i/o profile makes hbase run smoother. Over in
> HBASE-2457, good work has been done already describing the tension between
> minimizing compactions versus minimizing count of store files. This issue is
> about following on from what has been done in 2457 but also, breaking the
> hard-to-read compaction code out of Store.java out to a standalone class that
> can be the easier tested (and easily analyzed for its performance
> characteristics).
> If possible, in the refactor, we'd allow specification of alternate merge
> sort implementations.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
