[
https://issues.apache.org/jira/browse/HBASE-16454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428037#comment-15428037
]
ramkrishna.s.vasudevan commented on HBASE-16454:
------------------------------------------------
bq.I mean whether this minFilesToCompact is the number of files getting
compacted in this particular compaction. Or is it the min one (def to 3)
It is the one that is configured
{code}
long minFilesToCompact = Math.max(2L,
conf.getInt(CompactionConfiguration.HBASE_HSTORE_COMPACTION_MIN_KEY,
/* old name */ conf.getInt("hbase.hstore.compactionThreshold", 3)));
{code}
So sometimes if this value is also set to some higher value we may end up in
much bigger numbers.
> Compactor's shipping logic decision should be based on the current store's
> block size
> -------------------------------------------------------------------------------------
>
> Key: HBASE-16454
> URL: https://issues.apache.org/jira/browse/HBASE-16454
> Project: HBase
> Issue Type: Bug
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Priority: Minor
> Fix For: 2.0.0
>
>
> Currently we do call KeyValueScanner.shipped() if the size of the KVs
> compacted reaches certain threshold
> {code}
> if (kvs != null && bytesWrittenProgressForShippedCall >
> shippedCallSizeLimit) {
> kvs.shipped();
> bytesWrittenProgressForShippedCall = 0;
> }
> {code}
> Where the shippedCallSizeLimit is
> {code}
> long shippedCallSizeLimit = (long) minFilesToCompact *
> HConstants.DEFAULT_BLOCKSIZE;
> {code}
> Instead it is better if it is based on the actual configured block size for
> that store. It can still have the multiplying factore of 'minFilesToCompact '
> but having default store size would mean that if the store has smaller block
> sizes configured we hold on to more blocks and those will not get evicted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
