[
https://issues.apache.org/jira/browse/HBASE-29064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914279#comment-17914279
]
Duo Zhang commented on HBASE-29064:
-----------------------------------
I think the current replication framework does not have the problem. Once
return from the replicate method, we need to make sure that the wal entries has
been persistent. Replication is already asynchronous, so it is kinda OK to have
a synchronous operation here.
So the simpliest way to solve the problem here, is to just wait inside the
replicate method to make sure the entries has been persist to the backup
storage before returning from the method.
Thanks.
> POC: Implement Dual Offset Tracking for WAL Replication
> -------------------------------------------------------
>
> Key: HBASE-29064
> URL: https://issues.apache.org/jira/browse/HBASE-29064
> Project: HBase
> Issue Type: Task
> Reporter: Vinayak Hegde
> Assignee: Andor Molnar
> Priority: Major
> Labels: pull-request-available
>
> The current replication framework updates the WAL offset immediately after
> processing entries, which may lead to data loss in case of partial writes or
> failures before persisting the data to the backup storage (e.g., S3).
> To address this:
> # {*}Delivered Offset{*}: Tracks entries sent to the replication endpoint.
> # {*}Committed Offset{*}: Tracks entries fully persisted to the backup
> storage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
