[
https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269025#comment-13269025
]
Matt Corgan commented on HBASE-5930:
------------------------------------
Periodically flushing the memstore seems like a good feature to me. Could also
help clear out cold data from memory to make more room for bigger memstores on
regions that are actually being used.
A different solution to the underlying data loss issue might be to have a third
client setting for WAL writing: NONE, SYNC, and ASYNC. ASYNC would write the
data to a memory buffer, return success to the client, and another thread would
flush the buffer to the WAL. The WAL would ideally only lag a few seconds
behind the memstores, but some form of throttling would probably be needed.
> Periodically flush the Memstore?
> --------------------------------
>
> Key: HBASE-5930
> URL: https://issues.apache.org/jira/browse/HBASE-5930
> Project: HBase
> Issue Type: Improvement
> Reporter: Lars Hofhansl
> Priority: Minor
>
> A colleague of mine ran into an interesting issue.
> He inserted some data with the WAL disabled, which happened to fit in the
> aggregate Memstores memory.
> Two weeks later he a had problem with the HDFS cluster, which caused the
> region servers to abort. He found that his data was lost. Looking at the log
> we found that the Memstores were not flushed at all during these two weeks.
> Should we have an option to flush memstores periodically. There are obvious
> downsides to this, like many small storefiles, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
