[
https://issues.apache.org/jira/browse/HADOOP-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561247#action_12561247
]
Billy Pearson commented on HADOOP-2673:
---------------------------------------
just adding some stats about this problem
{code}
2008-01-22 01:07:31,944 DEBUG org.apache.hadoop.hbase.HStore: started
compaction of 432 files using
hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for
540402979/in_rank
2008-01-22 01:23:17,384 DEBUG org.apache.hadoop.hbase.HStore: moving
540402979/in_rank/8016207671941404930 in
hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to
540402979/in_rank/8587356094792327071 in
hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/in_rank
2008-01-22 01:23:22,605 DEBUG org.apache.hadoop.hbase.HStore: started
compaction of 328 files using
hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for
540402979/rank_visual
2008-01-22 01:23:31,248 DEBUG org.apache.hadoop.hbase.HStore: moving
540402979/rank_visual/4003180598508023113 in
hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to
540402979/rank_visual/5001502002807562425 in
hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for
540402979/rank_visual
2008-01-22 01:23:34,438 INFO org.apache.hadoop.hbase.HRegion: compaction
completed on region webdata,,1200970130265. Took 57mins, 10sec
2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: compaction for
HStore 540402979/size needed.
2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HRegion: 540402979/size
needs compaction
2008-01-22 01:23:34,439 INFO org.apache.hadoop.hbase.HRegion: starting
compaction on region webdata,,1200970130265
2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: started
compaction of 323 files using
hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for
540402979/size
{code}
after finished compacting 432 files on the last column the compaction took
57min and 10 sec in that time there was 323 more flushes.
> Flusher flushing back to back when it should not be
> ---------------------------------------------------
>
> Key: HADOOP-2673
> URL: https://issues.apache.org/jira/browse/HADOOP-2673
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/hbase
> Environment: hadoop svn 613782
> Reporter: Billy Pearson
> Fix For: 0.16.0
>
> Attachments: log.log
>
>
> settings:
> hbase.regionserver.optionalcacheflushinterval = 300000 (5 mins)
> hbase.hregion.memcache.flush.size = 67108864 (64MB)
> I am attaching a log for this everything works fine until the first flush of
> the memcache. but once it flushes the first time correctly it flushes back to
> back many times even when it does not exceed one of the two limits above.
> I have not found but one spot where we update the variable LastFlushTime
> except when the server starts so this could be the problem but I may be
> missing it in the code somewhere. If it is getting updated then we need to
> look if something could be blocking it from updating after a memcache flush.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
