[
https://issues.apache.org/jira/browse/HBASE-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878700#action_12878700
]
stack commented on HBASE-1025:
------------------------------
Talked to Kannan about comment in the replay log method around next:
{code}
I think the comment I added is not valid considering that logReader.next() is
working on split files. I think we should assume here that the split happens
successfully at the master, and that the split files should be clean at the end.
The case that I am not very certain about is what happens if master dies in the
middle of the split. Is the oldfile.log in a potentially invalid state? How do
we cleanly restart the split-- without the RS starting to work on a half-done
file?
In any case, in logReader.next() (in Store.java:doReconstructionLog()) I think
it is the right model to assume we have clean/valid log files; and the burden
of detecting partial files should be pushed to splitLog().
{code}
I'll remove the comment on commit but will open new issue to see if we can add
tests for case where splitter fails mid-write of files (new master should clean
up old stuff and restart splitting).
> Reconstruction log playback has no bounds on memory used
> --------------------------------------------------------
>
> Key: HBASE-1025
> URL: https://issues.apache.org/jira/browse/HBASE-1025
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Fix For: 0.21.0
>
> Attachments: 1025-v2.txt, 1025-v3.txt, 1025-v5.patch, 1025.txt
>
>
> Makes a TreeMap and just keeps adding edits without regard for size of edits
> applied; could cause OOME (I've not seen a definitive case though have seen
> an OOME around time of a reconstructionlog replay -- perhaps this the straw
> that broke the fleas antlers?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
