[
https://issues.apache.org/jira/browse/HBASE-21856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930701#comment-16930701
]
Bin Shi commented on HBASE-21856:
---------------------------------
[~apurtell], I want to make sure we're on the same page – if we don't let the
source to send batches sequentially, they could arrive at the the same
destination in the sink out of order, then the sink side needs to buffer the
arrived batches within a time window and sort the batches by the
non-consecutive sequence IDs. Because the sequence IDs aren't consecutive, we
might have some edits with smaller sequence IDs arriving behind the time window
if the time window isn't big enough, but bigger time window results in longer
latency and extra complexity. Is my understanding correct?
> Consider Causal Replication Ordering
> ------------------------------------
>
> Key: HBASE-21856
> URL: https://issues.apache.org/jira/browse/HBASE-21856
> Project: HBase
> Issue Type: Brainstorming
> Components: Replication
> Reporter: Lars Hofhansl
> Priority: Major
> Labels: Replication
>
> We've had various efforts to improve the ordering guarantees for HBase
> replication, most notably Serial Replication.
> I think in many cases guaranteeing a Total Replication Order is not required,
> but a simpler Causal Replication Order is sufficient.
> Specifically we would guarantee causal ordering for a single Rowkey. Any
> changes to a Row - Puts, Deletes, etc - would be replicated in the exact
> order in which they occurred in the source system.
> Unlike total ordering this can be accomplished with only local region server
> control.
> I don't have a full design in mind, let's discuss here. It should be
> sufficient to to the following:
> # RegionServers only adopt the replication queues from other RegionServers
> for regions they (now) own. This requires log splitting for replication.
> # RegionServers ship all edits for queues adopted from other servers before
> any of their "own" edits are shipped.
> It's probably a bit more involved, but should be much cheaper that the total
> ordering provided by serial replication.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
