[
https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727711#comment-14727711
]
Ashish Singhi commented on HBASE-13153:
---------------------------------------
bq. There seem to be two types of notifications from the peer cluster
No, only one. Sorry if I confused you.
At a very high level the sequence of operation will be something like below for
a positive cycle,
(1) doBulkLoad(HFiles, source cluster IDs)
(2) notify(BulkLoadListeners)
(3) ZK(add HFile entries only if no cyclic loop) \[return to the client\]
(4) ReplicateEntries(entries, source cluster ids) --> Back to step (1).
c1->c2
Complete bulk load process at cluster c1,
(1) doBulkLoad(HFiles)
(2) notify(BulkLoadListeners)
(3) ZK(add HFile entries, no cyclic loop -> replicate to c2) \[return to the
client\]
(4) ReplicateEntries(entries, source cluster ids)
Replication process at cluster c2,
(1) doBulkLoad(HFiles,{c1\})
(2) notify(BulkLoadListeners) \[return to the client, client will delete the
znode\]
For cyclic replication case,
c1->c2->c3->c1
Complete bulk load process at cluster c1,
(1) doBulkLoad(HFiles)
(2) notify(BulkLoadListeners)
(3) ZK(add HFile entries, no cyclic loop -> replicate to c2) \[return to the
client\]
(4) ReplicateEntries(entries, source cluster ids)
Replication process at cluster c2,
(1) doBulkLoad(HFiles,{c1\})
(2) notify(BulkLoadListeners)
(3) ZK(add HFile entries, no cyclic loop -> replicate to c3) \[return to the
client, client will delete the znode\]
(4) ReplicateEntries(entries, source cluster ids) --> Back to step (1).
Replication process at cluster c3,
(1) doBulkLoad(HFiles,{c1,c2\})
(2) notify(BulkLoadListeners)
(3) ZK(add HFile entries, cyclic loop detected) \[return to the client, client
will delete the znode\]
Cyclic loop ends here
Hope I am able to clear your questions.
> enable bulkload to support replication
> --------------------------------------
>
> Key: HBASE-13153
> URL: https://issues.apache.org/jira/browse/HBASE-13153
> Project: HBase
> Issue Type: New Feature
> Components: Replication
> Reporter: sunhaitao
> Assignee: Ashish Singhi
> Fix For: 2.0.0
>
> Attachments: HBase Bulk Load Replication.pdf
>
>
> Currently we plan to use HBase Replication feature to deal with disaster
> tolerance scenario.But we encounter an issue that we will use bulkload very
> frequently,because bulkload bypass write path, and will not generate WAL, so
> the data will not be replicated to backup cluster. It's inappropriate to
> bukload twice both on active cluster and backup cluster. So i advise do some
> modification to bulkload feature to enable bukload to both active cluster and
> backup cluster
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
