[
https://issues.apache.org/jira/browse/HBASE-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13685994#comment-13685994
]
stack commented on HBASE-8741:
------------------------------
bq. I think read the last "completed" WAL trailer to get the sequenceId at time
of rolling, and add a delta big enough to avoid reading the last (and most
probably incomplete) WAL file.
Chatting w/ [[email protected]], guessing a seqid for the last file would
be difficult in case where a new region was opened after the last WAL was
opened but before the server crashed. The new region could have a sequenceid
far in excess of the sequenceid that is the last-but-one WAL.
Chatting more, we were trying what to do here. Say 30 WALs. We could start
replaying WAL files at file 1. Meantime we could try recovering the last file.
Once recovered, we could read to the end of the file to get the last sequence
id and then pass this to the new regionserver host. It could then open the
region for Writes. It would be good if we did not have to read the file twice
-- once to get the sequenceid and then again to replay the edits.
In the multiWAL case, we could have multiple unfinished WALs. We would not be
able to open the region until all unfinished files had been recovered and we'd
figured the maximum sequenceid of all files. This could be tough to do in the
distributed replay case (How would we coordinate recovery of multiple
unfinished WALs and not open the region until all had been recovered?)
> Mutations on Regions in recovery mode might have same sequenceIDs
> -----------------------------------------------------------------
>
> Key: HBASE-8741
> URL: https://issues.apache.org/jira/browse/HBASE-8741
> Project: HBase
> Issue Type: Bug
> Components: MTTR
> Affects Versions: 0.95.1
> Reporter: Himanshu Vashishtha
> Assignee: Himanshu Vashishtha
>
> Currently, when opening a region, we find the maximum sequence ID from all
> its HFiles and then set the LogSequenceId of the log (in case the later is at
> a small value). This works good in recovered.edits case as we are not writing
> to the region until we have replayed all of its previous edits.
> With distributed log replay, if we want to enable writes while a region is
> under recovery, we need to make sure that the logSequenceId > maximum
> logSequenceId of the old regionserver. Otherwise, we might have a situation
> where new edits have same (or smaller) sequenceIds.
> We can store region level information in the WALTrailer, than this scenario
> could be avoided by:
> a) reading the trailer of the "last completed" file, i.e., last wal file
> which has a trailer and,
> b) completely reading the last wal file (this file would not have the
> trailer, so it needs to be read completely).
> In future, if we switch to multi wal file, we could read the trailer for all
> completed WAL files, and reading the remaining incomplete files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
