[
https://issues.apache.org/jira/browse/HBASE-17471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999500#comment-15999500
]
Hudson commented on HBASE-17471:
--------------------------------
SUCCESS: Integrated in Jenkins build HBase-1.4 #725 (See
[https://builds.apache.org/job/HBase-1.4/725/])
HBASE-17471 Region Seqid will be out of order in WAL if using (liyu: rev
9a97e28bdf202ea2a8cadffbb8f8de4c1411447e)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestWALObserver.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALFactory.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestFSHLog.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALActionsListener.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MultiVersionConcurrencyControl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSWALEntry.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSecureWAL.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALKey.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWALLockup.java
> Region Seqid will be out of order in WAL if using mvccPreAssign
> ---------------------------------------------------------------
>
> Key: HBASE-17471
> URL: https://issues.apache.org/jira/browse/HBASE-17471
> Project: HBase
> Issue Type: Bug
> Components: wal
> Affects Versions: 2.0.0, 1.4.0
> Reporter: Allan Yang
> Assignee: Allan Yang
> Priority: Critical
> Attachments: HBASE-17471-branch-1.v0.patch,
> HBASE-17471-branch-1.v1.patch, HBASE-17471-branch-1.v2.patch,
> HBASE-17471-branch-1.v3.patch, HBASE-17471-duo.patch,
> HBASE-17471-duo-v1.patch, HBASE-17471-duo-v2.patch, HBASE-17471.patch,
> HBASE-17471.tmp, HBASE-17471.v2.patch, HBASE-17471.v3.patch,
> HBASE-17471.v4.patch, HBASE-17471.v5.patch, HBASE-17471.v6.patch
>
>
> mvccPreAssign was brought by HBASE-16698, which truly improved the
> performance of writing, especially in ASYNC_WAL scenario. But mvccPreAssign
> was only used in {{doMiniBatchMutate}}, not in Increment/Append path. If
> Increment/Append and batch put are using against the same region in parallel,
> then seqid of the same region may not monotonically increasing in the WAL.
> Since one write path acquires mvcc/seqid before append, and the other
> acquires in the append/sync consume thread.
> The out of order situation can easily reproduced by a simple UT, which was
> attached in the attachment. I modified the code to assert on the disorder:
> {code}
> if(this.highestSequenceIds.containsKey(encodedRegionName)) {
> assert highestSequenceIds.get(encodedRegionName) < sequenceid;
> }
> {code}
> I'd like to say, If we allow disorder in WALs, then this is not a issue.
> But as far as I know, if {{highestSequenceIds}} is not properly set, some
> WALs may not archive to oldWALs correctly.
> which I haven't figure out yet is that, will disorder in WAL cause data loss
> when recovering from disaster? If so, then it is a big problem need to be
> fixed.
> I have fix this problem in our costom1.1.x branch, my solution is using
> mvccPreAssign everywhere, making it un-configurable. Since mvccPreAssign it
> is indeed a better way than assign seqid in the ringbuffer thread while
> keeping handlers waiting for it.
> If anyone think it is doable, then I will port it to branch-1 and master
> branch and upload it.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
