[
https://issues.apache.org/jira/browse/HBASE-10556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13903151#comment-13903151
]
Hadoop QA commented on HBASE-10556:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12629332/HBASE-10556-trunk_v1.patch
against trunk revision .
ATTACHMENT ID: 12629332
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8729//console
This message is automatically generated.
> Possible data loss due to non-handled DroppedSnapshotException for
> user-triggered flush from client/shell
> ---------------------------------------------------------------------------------------------------------
>
> Key: HBASE-10556
> URL: https://issues.apache.org/jira/browse/HBASE-10556
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Feng Honghua
> Assignee: Feng Honghua
> Priority: Critical
> Attachments: HBASE-10556-trunk_v1.patch
>
>
> During the code review when investigating HBASE-10499, a possibility of data
> loss due to non-handled DroppedSnapshotException for user-triggered flush is
> exposed.
> Data loss can happen as below:
> # A flush for some region is triggered via HBaseAdmin or shell
> # The request reaches regionserver and eventually HRegion.internalFlushcache
> is called, then fails at persisting memstore's snapshot to hfile,
> DroppedSnapshotException is thrown and the snapshot is left not cleared.
> # DroppedSnapshotException is not handled in HRegion, and is just
> encapsulated as a ServiceException before returning to client
> # After a while, some new writes are handled and put in the current memstore,
> then a new flush is triggered for the region due to memstoreSize exceeds
> flush threshold
> # This second(new) flush succeeds, for the HStore which failed in the
> previous user-triggered flush, the remained non-empty snapshot is used rather
> than a new snapshot made from the current memstore, but HLog's latest
> sequenceId is used for the resultant hfiles --- the sequenceId attached
> within the hfiles says all edits with sequenceId <= it have all been
> persisted, but actually it's not the truth for the edits still in the
> existing memstore
> # Now the regionserver hosting this region dies
> # During the replay phase of failover, the edits corresponding to the ones
> while are in memstore and not actually persisted in hfiles when the previous
> regionserver dies will be ignored, since they are deemed as persisted by
> compared to the hfiles' latest consequenceID --- These edits are lost...
> For the second flush, we also can't discard the remained snapshot and make a
> new one using current memstore, that way the data in the remained snapshot is
> lost. We should abort the regionserver immediately and rely on the failover
> to replay the log for data safety.
> DroppedSnapshotException is correctly handled in MemStoreFlusher for
> internally triggered flush (which are generated by flush-size / rollWriter /
> periodicFlusher). But user-triggered flush is processed directly by
> HRegionServer->HRegion without putting a flush entry to flushQueue, hence not
> handled by MemStoreFlusher
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
