[
https://issues.apache.org/jira/browse/HBASE-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-1025:
-------------------------
Attachment: 1025-v2.txt
So, replaying edits previously, we were not writing the WAL, we were not
filling in the new consistency ts, we were not keeping account of size so we
knew when to flag flush, etc.
This patch calls HRegion#put and HRegion#delete adding the edits from the
replay log rather than add edits direct to memstore (w/o an accounting in WAL
so that is we successfully played all edits but crashed before we flushed, we'd
lose the edits permanently). Calling HRegion#put and delete is profligate in
object creation but it'll be more 'correct'. It also triggers flush if
memstore grows too large.
I added to test assertion flushes are happening and that deletes make it across
splits.
We are losing edits though, even when number is small. Its probably some
timestamping issue. Will figure it out.
Also need to figure what to do if we crash mid-replay AFTER a flush has
happened. Might worry about this in separate issue.
> Reconstruction log playback has no bounds on memory used
> --------------------------------------------------------
>
> Key: HBASE-1025
> URL: https://issues.apache.org/jira/browse/HBASE-1025
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Kannan Muthukkaruppan
> Fix For: 0.21.0
>
> Attachments: 1025-v2.txt, 1025.txt
>
>
> Makes a TreeMap and just keeps adding edits without regard for size of edits
> applied; could cause OOME (I've not seen a definitive case though have seen
> an OOME around time of a reconstructionlog replay -- perhaps this the straw
> that broke the fleas antlers?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
