[
https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565210#action_12565210
]
Billy Pearson commented on HADOOP-2636:
---------------------------------------
sense HADOOP-2731 was committed hbase has been much more stable on my test.
flushes are working correctly now. if we could build off that and set flusher
to only flush columns that need to be flushed (have data to flush).
I been thanking about the option flusher and we might need to leave it in there
to help keep memory usage lower on a region server if we left it we would be
able to remove the regions from memory that have got updates but have not got
the amount needed for a flush. This would be useful for users with 32bit cpus
and are limited by the 2000MB memory usage and users with limited memory. maybe
just leave it in there at 30 mins or 60 mins so it will clean up memory usage
once an hour per region if needed. Setting to 30-60 mins would not impact the
compactor as much if we are only flushing regions columns that have got updates.
> [hbase] Make cache flush triggering less simplistic
> ---------------------------------------------------
>
> Key: HADOOP-2636
> URL: https://issues.apache.org/jira/browse/HADOOP-2636
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.16.0
> Reporter: stack
> Assignee: Jim Kellerman
> Fix For: 0.17.0
>
> Attachments: patch.txt, patch.txt, patch.txt, patch.txt, patch.txt,
> patch.txt, patch.txt, patch.txt
>
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a
> configurable max size -- we flush all Stores though a Store memcache might
> have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some
> substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those
> Stores > 50% of max memcache size. Behavior would vary dependent on the
> prompt that provoked the flush. Would also log why the flush is running:
> optional or > max size.
> This issue comes out of HADOOP-2621.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
