[
https://issues.apache.org/jira/browse/HBASE-10201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199822#comment-14199822
]
zhangduo commented on HBASE-10201:
----------------------------------
Results on master branch
2.0.0-SNAPSHOT, revision=ecd708671c135052a175c88603d5215a0434e4fa
metric_storeCount: 3,
metric_storeFileCount: 9,
metric_memStoreSize: 40117320,
metric_storeFileSize: 4461018704,
metric_compactionsCompletedCount: 92,
metric_numBytesCompactedCount: 22091556672,
metric_numFilesCompactedCount: 290
Write amplification(numBytesCompactedCount/storeFileSize): 4.95
Elapsed time: 23m32s
2.0.0-SNAPSHOT, revision=ecd708671c135052a175c88603d5215a0434e4fa with
HBASE-10201
metric_storeCount: 3,
metric_storeFileCount: 8,
metric_memStoreSize: 16400424,
metric_storeFileSize: 4483028246,
metric_compactionsCompletedCount: 54,
metric_numBytesCompactedCount: 20497293164,
metric_numFilesCompactedCount: 178
Write amplification(numBytesCompactedCount/storeFileSize): 4.57
Elapsed time: 23m5s
2.0.0-SNAPSHOT, revision=ecd708671c135052a175c88603d5215a0434e4fa with
HBASE-10201 but disable selective flush
metric_storeCount: 3,
metric_storeFileCount: 9,
metric_memStoreSize: 39937056,
metric_storeFileSize: 4461185232,
metric_compactionsCompletedCount: 92,
metric_numBytesCompactedCount: 22092540348,
metric_numFilesCompactedCount: 290
Write amplification(numBytesCompactedCount/storeFileSize): 4.95
Elapsed time: 22m51s
Seems default config on master will do compactions more aggresive, but the
result of WAF decrease is not changed too much.
(4.95-4.57)/4.95=7.68%
> Port 'Make flush decisions per column family' to trunk
> ------------------------------------------------------
>
> Key: HBASE-10201
> URL: https://issues.apache.org/jira/browse/HBASE-10201
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Reporter: Ted Yu
> Assignee: zhangduo
> Priority: Critical
> Fix For: 2.0.0, 0.99.2
>
> Attachments: 3149-trunk-v1.txt, HBASE-10201-0.98.patch,
> HBASE-10201-0.98_1.patch, HBASE-10201-0.98_2.patch, HBASE-10201-0.99.patch,
> HBASE-10201.patch, HBASE-10201_1.patch, HBASE-10201_2.patch,
> HBASE-10201_3.patch, HBASE-10201_4.patch, HBASE-10201_5.patch
>
>
> Currently the flush decision is made using the aggregate size of all column
> families. When large and small column families co-exist, this causes many
> small flushes of the smaller CF. We need to make per-CF flush decisions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
