[
https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj Das updated HBASE-5930:
-------------------------------
Attachment: 5930-2.2.patch
[~lhofhansl], I think the patch addresses the use case you mention. The
shouldFlush call in HRegion first does a global check whether there was a
recent flush done, and if so, returns. If not, it goes over the memstores and
if there is a memstore edit that happened after the last flush (and due to the
previous global check, implicitly after the timeinterval), it returns true
indicating that a flush is needed now. In general, if a memstore has at least
one edit after the last flush, and the last flush happened before the
timeinterval (let's say configured to 60 minutes), then a flush will be
triggered now. The flushes are offset by a random sleep related up to a max of
2 minutes.
The attached patch addresses a few of Ted's comments (setting the interrupt
status back in the PeriodicMemstoreFlusher chore). It also puts a minimum delay
of 1 sec between flushed. It makes the default sleep to 60 minutes.
[~lhofhansl], would appreciate if you could take a look at this please, and let
me know if I missed anything in the patch w.r.t the scenario you outlined..
> Periodically flush the Memstore?
> --------------------------------
>
> Key: HBASE-5930
> URL: https://issues.apache.org/jira/browse/HBASE-5930
> Project: HBase
> Issue Type: Improvement
> Reporter: Lars Hofhansl
> Assignee: Devaraj Das
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5930-1.patch, 5930-2.1.patch, 5930-2.2.patch,
> 5930-wip.patch
>
>
> A colleague of mine ran into an interesting issue.
> He inserted some data with the WAL disabled, which happened to fit in the
> aggregate Memstores memory.
> Two weeks later he a had problem with the HDFS cluster, which caused the
> region servers to abort. He found that his data was lost. Looking at the log
> we found that the Memstores were not flushed at all during these two weeks.
> Should we have an option to flush memstores periodically. There are obvious
> downsides to this, like many small storefiles, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
