[
https://issues.apache.org/jira/browse/HBASE-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13691620#comment-13691620
]
Himanshu Vashishtha commented on HBASE-8741:
--------------------------------------------
So, the basic scheme looks like:
A region carries a sequenceID counter which will be used while inserting
entries in the WAL.
On opening a region, it will be set to whatever the max flush sequenceId of the
region is (obtained from the hfiles). A WAL will have entries from various
regions (so it is not in increasing order now; that should be ok). When we roll
the wal, we keep a data structure of <WAL_file_path> :
(<Region>:<Current_value_of_RegionSequenceId>>).
While archiving WALs, we need to make sure that we archive wals if all its
entries are already flushed. We would compare the above data structure
(basically a map of region to sequenceId at time of rolling) to the
max(completeSequenceId, openSequenceId) for each region. If it is less, then we
no longer need that wal, and archive it; else we submit a flush request for
those regions whose entries are greater.
For doing writes doing failover, we need to give a hint to the new regionserver
some value (could be equal to current number of wals * max number of entries a
wal could take). This should be used to bump up the value of sequenceId counter
of the region.
> Mutations on Regions in recovery mode might have same sequenceIDs
> -----------------------------------------------------------------
>
> Key: HBASE-8741
> URL: https://issues.apache.org/jira/browse/HBASE-8741
> Project: HBase
> Issue Type: Bug
> Components: MTTR
> Affects Versions: 0.95.1
> Reporter: Himanshu Vashishtha
> Assignee: Himanshu Vashishtha
>
> Currently, when opening a region, we find the maximum sequence ID from all
> its HFiles and then set the LogSequenceId of the log (in case the later is at
> a small value). This works good in recovered.edits case as we are not writing
> to the region until we have replayed all of its previous edits.
> With distributed log replay, if we want to enable writes while a region is
> under recovery, we need to make sure that the logSequenceId > maximum
> logSequenceId of the old regionserver. Otherwise, we might have a situation
> where new edits have same (or smaller) sequenceIds.
> We can store region level information in the WALTrailer, than this scenario
> could be avoided by:
> a) reading the trailer of the "last completed" file, i.e., last wal file
> which has a trailer and,
> b) completely reading the last wal file (this file would not have the
> trailer, so it needs to be read completely).
> In future, if we switch to multi wal file, we could read the trailer for all
> completed WAL files, and reading the remaining incomplete files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
