[
https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562954#action_12562954
]
viper799 edited comment on HADOOP-2636 at 1/26/08 8:49 PM:
----------------------------------------------------------------
I tried your patch out above and it only flushing one column for me I have
three getting data and I only see one flushing and its flushing back to back
over and over.
Example this is a flush on the same column 4 times with in one sec
{code}
2008-01-26 22:40:14,137 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache
flush for region webdata,,1201405676281 store 332212182/in_rank
2008-01-26 22:40:14,418 DEBUG org.apache.hadoop.hbase.HStore: Added
332212182/in_rank/1595847912559744983 with 70 entries, sequence id 1877650, and
size 6.6k for 332212182/in_rank
2008-01-26 22:40:14,418 DEBUG org.apache.hadoop.hbase.HRegion: Finished
memcache flush for store 332212182/in_rank in 281ms, sequenceid=1877650
2008-01-26 22:40:14,436 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache
flush for region webdata,,1201405676281 store 332212182/in_rank
2008-01-26 22:40:14,650 DEBUG org.apache.hadoop.hbase.HStore: Added
332212182/in_rank/3253290776281930479 with 6 entries, sequence id 1877667, and
size 621.0 for 332212182/in_rank
2008-01-26 22:40:14,650 DEBUG org.apache.hadoop.hbase.HRegion: Finished
memcache flush for store 332212182/in_rank in 214ms, sequenceid=1877667
2008-01-26 22:40:14,682 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache
flush for region webdata,,1201405676281 store 332212182/in_rank
2008-01-26 22:40:14,893 DEBUG org.apache.hadoop.hbase.HStore: Added
332212182/in_rank/6244850576092789885 with 5 entries, sequence id 1877683, and
size 497.0 for 332212182/in_rank
2008-01-26 22:40:14,894 DEBUG org.apache.hadoop.hbase.HRegion: Finished
memcache flush for store 332212182/in_rank in 212ms, sequenceid=1877683
2008-01-26 22:40:14,941 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache
flush for region webdata,,1201405676281 store 332212182/in_rank
{code}
was (Author: viper799):
I tried your patch out above and it only flushing one column for me I have
three getting date and I only see one flushing and its flushing back to back
over and over.
example this is a flush on the same column 4 times with in one sec
{code}
2008-01-26 22:40:14,137 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache
flush for region webdata,,1201405676281 store 332212182/in_rank
2008-01-26 22:40:14,418 DEBUG org.apache.hadoop.hbase.HStore: Added
332212182/in_rank/1595847912559744983 with 70 entries, sequence id 1877650, and
size 6.6k for 332212182/in_rank
2008-01-26 22:40:14,418 DEBUG org.apache.hadoop.hbase.HRegion: Finished
memcache flush for store 332212182/in_rank in 281ms, sequenceid=1877650
2008-01-26 22:40:14,436 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache
flush for region webdata,,1201405676281 store 332212182/in_rank
2008-01-26 22:40:14,650 DEBUG org.apache.hadoop.hbase.HStore: Added
332212182/in_rank/3253290776281930479 with 6 entries, sequence id 1877667, and
size 621.0 for 332212182/in_rank
2008-01-26 22:40:14,650 DEBUG org.apache.hadoop.hbase.HRegion: Finished
memcache flush for store 332212182/in_rank in 214ms, sequenceid=1877667
2008-01-26 22:40:14,682 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache
flush for region webdata,,1201405676281 store 332212182/in_rank
2008-01-26 22:40:14,893 DEBUG org.apache.hadoop.hbase.HStore: Added
332212182/in_rank/6244850576092789885 with 5 entries, sequence id 1877683, and
size 497.0 for 332212182/in_rank
2008-01-26 22:40:14,894 DEBUG org.apache.hadoop.hbase.HRegion: Finished
memcache flush for store 332212182/in_rank in 212ms, sequenceid=1877683
2008-01-26 22:40:14,941 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache
flush for region webdata,,1201405676281 store 332212182/in_rank
{code}
> [hbase] Make cache flush triggering less simplistic
> ---------------------------------------------------
>
> Key: HADOOP-2636
> URL: https://issues.apache.org/jira/browse/HADOOP-2636
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Jim Kellerman
> Fix For: 0.17.0
>
> Attachments: patch.txt
>
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a
> configurable max size -- we flush all Stores though a Store memcache might
> have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some
> substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those
> Stores > 50% of max memcache size. Behavior would vary dependent on the
> prompt that provoked the flush. Would also log why the flush is running:
> optional or > max size.
> This issue comes out of HADOOP-2621.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
