[
https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977841#comment-14977841
]
Heng Chen commented on HBASE-14004:
-----------------------------------
{quote}
What if the regionserver crashed before flushing HFile? I think the record will
come back since it has already been persisted in WAL.
{quote}
Indeed, as current logic, when sync failed, the memstore will rollback, and
client will be told 'write failed'.
And if RS crash before memstore flush, the record will came back after WAL
replayed. So client will found write action is not failed, it is inconsistent!
{quote}
Add a marker maybe a solution, but you need to check the marker everywhere when
replaying WAL, and you still need to deal with the failure when placing
marker...
{quote}
Agreed!
Thanks [~Apache9] for your reply!
> [Replication] Inconsistency between Memstore and WAL may result in data in
> remote cluster that is not in the origin
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-14004
> URL: https://issues.apache.org/jira/browse/HBASE-14004
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: He Liangliang
> Priority: Critical
> Labels: replication, wal
>
> Looks like the current write path can cause inconsistency between
> memstore/hfile and WAL which cause the slave cluster has more data than the
> master cluster.
> The simplified write path looks like:
> 1. insert record into Memstore
> 2. write record to WAL
> 3. sync WAL
> 4. rollback Memstore if 3 fails
> It's possible that the HDFS sync RPC call fails, but the data is already
> (may partially) transported to the DNs which finally get persisted. As a
> result, the handler will rollback the Memstore and the later flushed HFile
> will also skip this record.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
