[
https://issues.apache.org/jira/browse/HBASE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902206#action_12902206
]
Nicolas Spiegelberg commented on HBASE-2643:
--------------------------------------------
IIIRC:
[5:09pm] nspiegelberg: I'm just wondering if there's a reason for us to not
continue through an EOF
[5:09pm] St^Ack: I can't think of one
[5:10pm] St^Ack: what you say is reasonable nspiegelberg
[5:10pm] St^Ack: least till we learn otherwise
[5:10pm] St^Ack: I have to go lads... pick up kid
[5:10pm] kannan: i agree... we could check to see if the IOE was an EOF and
continue on....
> Figure how to deal with eof splitting logs
> ------------------------------------------
>
> Key: HBASE-2643
> URL: https://issues.apache.org/jira/browse/HBASE-2643
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Nicolas Spiegelberg
> Priority: Blocker
> Fix For: 0.90.0
>
>
> When splitting the WAL and encountering EOF, it's not clear what to do.
> Initial discussion of this started in http://review.hbase.org/r/74/ -
> summarizing here for brevity:
> We can get an EOFException while splitting the WAL in the following cases:
> - The writer died after creating the file but before even writing the header
> (or crashed halfway through writing the header)
> - The writer died in the middle of flushing some data - sync() guarantees
> that we can see _at least_ the last edit, but we may see half of an edit that
> was being written out when the RS crashed (especially for large rows)
> - The data was actually corrupted somehow (eg a length field got changed to
> be too long and thus points past EOF)
> Ideally we would know when we see EOF whether it was really the last record,
> and in that case, simply drop that record (it wasn't synced, so therefore we
> dont need to split it). Some open questions:
> - Currently we ignore empty files. Is it ok to ignore an empty log file if
> it's not the last one?
> - Similarly, do we ignore an EOF mid-record if it's not the last log file?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
