@Demai, We actually have 10 clusters in different locations. The replication scope is not an issue for me since I have only one column family and we want it replicated to each location. Can you elaborate more on why a replication setup of more than 3-4 clusters would be a headache in your opinion?
On Fri, Nov 8, 2013 at 2:16 PM, Ishan Chhabra <[email protected]>wrote: > @Demai, > Writes from B should also go to A and C. So, if I were to continue on your > suggestion, I would setup A-B master master and B-C master-master, which is > what I was proposing in the 2nd approach (MST based). > > @Vladimir > That is classified. :P > > > On Fri, Nov 8, 2013 at 1:20 PM, Vladimir Rodionov > <[email protected]>wrote: > >> *I want to setup NxN replication i.e. N clusters each replicating to each >> other. N is expected to be around 10.* >> >> Preparing to thermonuclear war? >> >> >> >> >> On Fri, Nov 8, 2013 at 1:14 PM, Ishan Chhabra <[email protected] >> >wrote: >> >> > I want to setup NxN replication i.e. N clusters each replicating to each >> > other. N is expected to be around 10. >> > >> > On doing some research, I realize it is possible after HBASE-7709 fix, >> but >> > it would lead to much more data flowing in the system. eg. >> > >> > Lets say we have 3 clusters: A,B and C. >> > A new write to A will go to B and then C, and also go to C directly via >> the >> > direct path. This leads to unnecessary network usage and writes to WAL >> of >> > B, that should be avoided. Now imagine this with 10 clusters, it won’t >> > scale. >> > >> > One option is to create a minimum spanning tree joining all the clusters >> > and make nodes replicate to their immediate peers in a master-master >> > fashion. This is much better than NxN mesh, but still has extra network >> and >> > WAL usage. It also suffers from a failure scenarios where the a single >> > cluster going down will pause replication to clusters downstream. >> > >> > What I really want is that the ReplicationSource should only forward >> > WALEdits with cluster-id same as the local cluster-id. This seems like a >> > straight forward patch to put in. >> > >> > Any thoughts on the suggested approach or alternatives? >> > >> > -- >> > *Ishan Chhabra *| Rocket Scientist | RocketFuel Inc. >> > >> > > > > -- > *Ishan Chhabra *| Rocket Scientist | RocketFuel Inc. > -- *Ishan Chhabra *| Rocket Scientist | RocketFuel Inc.
