[
https://issues.apache.org/jira/browse/HADOOP-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561241#action_12561241
]
jimk edited comment on HADOOP-2674 at 1/22/08 12:15 AM:
-----------------------------------------------------------------
I think the reason it doesn't add up is that there is either an optional cache
flush going on or the region is being closed during a split. When a region is
closed, what gets flushed is what ever is in the cache at the time.
In the current implementation, the caches are flushed when the total for all
columns reaches 64M. The only other callers of cache flush are the optional
timed cache flush and when a region is closed.
Optional cache flushing and zero length flushes will be removed in HADOOP-2636.
Further, since cache flushing will be on a per column basis, cache flushing
will only occur when the cache for that column reaches 64M.
was (Author: jimk):
I think the reason it doesn't add up is that there is either an optional
cache flush going on or the region is being closed during a split. When a
region is closed, what gets flushed is what ever is in the cache at the time.
Since the caching is done on a per column basis, each column can have up to 64M
cached.
Optional cache flushing and zero length flushes will be removed in HADOOP-2636.
> memcache size reported in debug log doesn ot add up
> ---------------------------------------------------
>
> Key: HADOOP-2674
> URL: https://issues.apache.org/jira/browse/HADOOP-2674
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/hbase
> Environment: hadoop svn 613782
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.16.0
>
>
> below shows a memcache size of 64M but when you add up the regions I get
> 149.3K = 152883.2B
> 153B = 153B
> 153B = 153B
> 3.8M = 3984588.8B
> 2.6M = 2726297.6B
> 153B = 153B
> 129K = 132096B
> 2.6M = 2726297.6B
> 153B = 153B
> total = 9569891.4B = 9.12M not 64M
> {code}
> 2008-01-21 23:51:30,812 DEBUG org.apache.hadoop.hbase.HRegion: Started
> memcache flush for region webdata,,1200970130265. Size 64.0m
> 2008-01-21 23:51:31,694 DEBUG org.apache.hadoop.hbase.HStore: Added
> 540402979/size/2337100008775916241 with 11004 entries, sequence id 32535827,
> and size 149.3k for 540402979/size
> 2008-01-21 23:51:31,956 DEBUG org.apache.hadoop.hbase.HStore: Added
> 540402979/rank_total/6511354828803004572 with 0 entries, sequence id
> 32535827, and size 153.0 for 540402979/rank_total
> 2008-01-21 23:51:32,210 DEBUG org.apache.hadoop.hbase.HStore: Added
> 540402979/first_seen/116626538562002170 with 0 entries, sequence id 32535827,
> and size 153.0 for 540402979/first_seen
> 2008-01-21 23:51:40,084 DEBUG org.apache.hadoop.hbase.HStore: Added
> 540402979/anchor/5635668875929221096 with 189726 entries, sequence id
> 32535827, and size 3.8m for 540402979/anchor
> 2008-01-21 23:51:48,134 DEBUG org.apache.hadoop.hbase.HStore: Added
> 540402979/last_seen/5152520466951569025 with 189726 entries, sequence id
> 32535827, and size 2.6m for 540402979/last_seen
> 2008-01-21 23:51:48,400 DEBUG org.apache.hadoop.hbase.HStore: Added
> 540402979/source/7366562139108336733 with 0 entries, sequence id 32535827,
> and size 153.0 for 540402979/source
> 2008-01-21 23:51:49,075 DEBUG org.apache.hadoop.hbase.HStore: Added
> 540402979/stime/5227996581170475543 with 11004 entries, sequence id 32535827,
> and size 129.6k for 540402979/stime
> 2008-01-21 23:51:55,944 DEBUG org.apache.hadoop.hbase.HStore: Added
> 540402979/in_rank/7235419568042410782 with 189726 entries, sequence id
> 32535827, and size 2.6m for 540402979/in_rank
> 2008-01-21 23:51:56,217 DEBUG org.apache.hadoop.hbase.HStore: Added
> 540402979/rank_visual/5975562391477267376 with 0 entries, sequence id
> 32535827, and size 153.0 for 540402979/rank_visual
> 2008-01-21 23:51:56,217 DEBUG org.apache.hadoop.hbase.HRegion: Finished
> memcache flush for region webdata,,1200970130265 in 25405ms,
> sequenceid=32535827
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
