[
https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642615#comment-13642615
]
Devaraj Das commented on HBASE-5930:
------------------------------------
bq. I'd like to open a separate issue on the queuing of flush requests
After some deliberations and code browsing, it seems like we don't need to do
this. The reasons being:
1. We already have the emergency flush procedure in place
(MemStoreFlusher.flushRegion with an emergencyFlush boolean argument exists
that is called when there is an urgent need to flush).
2. In the normal steady state of affairs, entries from the queue are processed
asynchronously anyway. Flushes are done by a thread and there is no guarantee
when the thread picks a flush request up. By adding a delay, this situation is
not worsened much, and especially in combination with (1) above.
3. To address the issue raised, I was thinking that when a new flush request
comes without a delay, we could remove an existing delayed-flush-request entry
that was queued before, and queue a new one without a delay. However, there is
a complication here - there is logic that queues up entries with delayed
flushes in _MemStoreFlusher.flushRegion(final FlushRegionEntry)_. That would be
impacted if I did what I said.
All in all, this doesn't seem like an issue.. [~enis] also ack'ed this in our
offline discussions.
> Limits the amount of time an edit can live in the memstore.
> -----------------------------------------------------------
>
> Key: HBASE-5930
> URL: https://issues.apache.org/jira/browse/HBASE-5930
> Project: HBase
> Issue Type: Improvement
> Reporter: Lars Hofhansl
> Assignee: Devaraj Das
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: 5930-0.94.txt, 5930-1.patch, 5930-2.1.patch,
> 5930-2.2.patch, 5930-2.3.patch, 5930-2.4.patch, 5930-track-oldest-sample.txt,
> 5930-wip.patch, HBASE-5930-ADD-0.patch, hbase-5930-addendum2.patch,
> hbase-5930-test-execution.log
>
>
> A colleague of mine ran into an interesting issue.
> He inserted some data with the WAL disabled, which happened to fit in the
> aggregate Memstores memory.
> Two weeks later he a had problem with the HDFS cluster, which caused the
> region servers to abort. He found that his data was lost. Looking at the log
> we found that the Memstores were not flushed at all during these two weeks.
> Should we have an option to flush memstores periodically. There are obvious
> downsides to this, like many small storefiles, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
