[
https://issues.apache.org/jira/browse/HBASE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626873#action_12626873
]
Billy Pearson commented on HBASE-834:
-------------------------------------
ok latest version of patch
834-0.2.1-patchv3.txt
Changed force to majorCompaction better name.
Passed majorCompaction down to compactHStoreFiles function so we could know not
to remove > max version on a minor compaction / incremental compaction
This should solve the problem with HBASE-826.
Stack said we do not need to max versions or expiration code on a minor
compaction but I left the expiration code the same
because if the data is passed its ttl it will not matter if its a minor or a
major compaction from what I can reason but I might be wrong let me know if
that needs to be changed also.
So as of now it will be removed so we do not have to read it again and remove
it on the next compaction = less work later theory.
looks like there is a bug in the rest/Dispatcher.java file current branch-0.2
will not compile clean but I thank my patch will build clean if that error is
fixed.
Yes the minor compaction has the limit of hbase.hstore.compaction.max
and majorcompactions do not have this limit.
> Upper bound on files we compact at any one time
> -----------------------------------------------
>
> Key: HBASE-834
> URL: https://issues.apache.org/jira/browse/HBASE-834
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.2.1, 0.18.0
> Reporter: stack
> Assignee: Billy Pearson
> Priority: Blocker
> Fix For: 0.2.1, 0.18.0
>
> Attachments: 834-0.2.1-patch.txt, 834-0.2.1-patchv2.txt,
> 834-0.2.1-patchv3.txt, 834-patch.txt
>
>
> From Billy in HBASE-64, which we closed because it got pulled all over the
> place:
> {code}
> Currently we do compaction on a region when the
> hbase.hstore.compactionThreshold is reached - default 3
> I thank we should configure a max number of mapfiles to compact at one time
> simulator to doing a minor compaction in bigtable. This keep compaction's
> form getting tied up in one region to long letting other regions get way to
> many memcache flushes making compaction take longer and longer for each region
> If we did that when a regions updates start to slack off the max number will
> eventuly include all mapfiles causeing a major compaction on that region.
> Unlike big table this would leave the master out of the process and letting
> the region server handle the major compaction when it has time.
> When doing a minor compaction on a few files I thank we should compact the
> newest mapfiles first leave the larger/older ones for when we have low
> updates to a region.
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
