[
https://issues.apache.org/jira/browse/HBASE-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406881#comment-13406881
]
Hadoop QA commented on HBASE-6311:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12535148/HBASE-6311v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 5 javac compiler warnings (more than
the trunk's current 4 warnings).
-1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.replication.TestReplication
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2325//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2325//console
This message is automatically generated.
> Data error after majorCompaction caused by keeping MVCC for opened scanners
> ---------------------------------------------------------------------------
>
> Key: HBASE-6311
> URL: https://issues.apache.org/jira/browse/HBASE-6311
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.94.0
> Reporter: chunhui shen
> Assignee: chunhui shen
> Priority: Blocker
> Fix For: 0.96.0, 0.94.1
>
> Attachments: HBASE-6311-test.patch, HBASE-6311v1.patch,
> HBASE-6311v2.patch
>
>
> It is a big problem we found in 0.94, and you could reproduce the problem in
> Trunk using the test case I uploaded.
> When we do compaction, we will use region.getSmallestReadPoint() to keep MVCC
> for opened scanners;
> However,It will make data mistake after majorCompaction because we will skip
> delete type KV but keep the put type kv in the compacted storefile.
> The following is the reason from code:
> In StoreFileScanner, enforceMVCC is false when compaction, so we could read
> the delete type KV,
> However, we will skip this delete type KV in ScanQueryMatcher because
> following code
> {code}
> if (kv.isDelete())
> {
> ...
> if (includeDeleteMarker
> && kv.getMemstoreTS() <= maxReadPointToTrackVersions) {
> System.out.println("add deletes,maxReadPointToTrackVersions="
> + maxReadPointToTrackVersions);
> this.deletes.add(bytes, offset, qualLength, timestamp, type);
> }
> ...
> }
> {code}
> Here maxReadPointToTrackVersions = region.getSmallestReadPoint();
> and kv.getMemstoreTS() > maxReadPointToTrackVersions
> So we won't add this to DeleteTracker.
> Why test case passed if remove the line
> MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint);
> Because in the StoreFileScanner#skipKVsNewerThanReadpoint
> {code}
> if (cur.getMemstoreTS() <= readPoint) {
> cur.setMemstoreTS(0);
> }
> {code}
> So if we remove the line
> MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint);
> Here readPoint is LONG.MAX_VALUE, we will set memStore ts as 0, so we will
> add it to DeleteTracker in ScanQueryMatcher
> Solution:
> We use smallestReadPoint of region when compaction to keep MVCC for OPENED
> scanner, So we should retain delete type kv in output in the case(Already
> deleted KV is retained in output to make old opened scanner could read this
> KV) even if it is a majorcompaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
