[
https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640745#comment-13640745
]
Lars Hofhansl edited comment on HBASE-5930 at 4/24/13 6:19 PM:
---------------------------------------------------------------
Sorry if I seem difficult with this one... How about a theme like this:
* We record the time of the first edit made to the memstore after a flush. We
can even improve this and only record the time of the first unlogged edit made.
* Periodically we run the chore, if the recorded time of that first edit is
older than a configurable X then we flush the memstore.
That would:
# be simpler
# clearly limit the maximum an edit will stay in the memstore without being
flushed
was (Author: lhofhansl):
Sorry if I seem difficult with this one... How about a theme like this:
* We record the time of the first edit made to the memstore since the last
flush. We can even improve this and only record the time of the last unlogged
edit made.
* Periodically we run the chore, if the recorded time of that first edit is
older than a configurable X then we flush the memstore.
That would:
# be simpler
# clearly limit the maximum an edit will stay in the memstore without being
flushed
> Periodically flush the Memstore?
> --------------------------------
>
> Key: HBASE-5930
> URL: https://issues.apache.org/jira/browse/HBASE-5930
> Project: HBase
> Issue Type: Improvement
> Reporter: Lars Hofhansl
> Assignee: Devaraj Das
> Priority: Minor
> Fix For: 0.95.1
>
> Attachments: 5930-1.patch, 5930-2.1.patch, 5930-2.2.patch,
> 5930-2.3.patch, 5930-2.4.patch, 5930-wip.patch
>
>
> A colleague of mine ran into an interesting issue.
> He inserted some data with the WAL disabled, which happened to fit in the
> aggregate Memstores memory.
> Two weeks later he a had problem with the HDFS cluster, which caused the
> region servers to abort. He found that his data was lost. Looking at the log
> we found that the Memstores were not flushed at all during these two weeks.
> Should we have an option to flush memstores periodically. There are obvious
> downsides to this, like many small storefiles, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
