[
https://issues.apache.org/jira/browse/HADOOP-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562335#action_12562335
]
stack commented on HADOOP-2615:
-------------------------------
Doing some load testing, I'm seeing compactions taking longer and longer as
reported above but I'm also seeing that the region won't split. Just goes
from one compaction to the next w/ each doing more and more files taking longer
each time.
{code}
2008-01-25 01:16:45,850 INFO org.apache.hadoop.hbase.HRegion: compaction
completed on region TestTable,,1201223677355. Took 55sec
2008-01-25 01:18:37,347 INFO org.apache.hadoop.hbase.HRegion: compaction
completed on region TestTable,,1201223677355. Took 1mins, 49sec
2008-01-25 01:21:42,010 INFO org.apache.hadoop.hbase.HRegion: compaction
completed on region TestTable,,1201223677355. Took 3mins, 4sec
2008-01-25 01:27:20,417 INFO org.apache.hadoop.hbase.HRegion: compaction
completed on region TestTable,,1201223677355. Took 5mins, 38sec
2008-01-25 01:37:55,330 INFO org.apache.hadoop.hbase.HRegion: compaction
completed on region TestTable,,1201223677355. Took 10mins, 34sec
{code}
Looking more into this.
> Add max number of mapfiles to compact at one time giveing us a minor & major
> compaction
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-2615
> URL: https://issues.apache.org/jira/browse/HADOOP-2615
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.17.0
>
>
> Currently we do compaction on a region when the
> hbase.hstore.compactionThreshold is reached - default 3
> I thank we should configure a max number of mapfiles to compact at one time
> simulator to doing a minor compaction in bigtable. This keep compaction's
> form getting tied up in one region to long letting other regions get way to
> many memcache flushes making compaction take longer and longer for each region
> If we did that when a regions updates start to slack off the max number will
> eventuly include all mapfiles causeing a major compaction on that region.
> Unlike big table this would leave the master out of the process and letting
> the region server handle the major compaction when it has time.
> When doing a minor compaction on a few files I thank we should compact the
> newest mapfiles first leave the larger/older ones for when we have low
> updates to a region.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
