Vlad, nice one. :-) Very good point. Unless your company have data center in 10 different locations, I don't see a good use case for such complex setup Also, please keep in mind, currently the only replication type is 'global'. Before we find a good way to specify peer at table:family level. It will be a nightmare to replicating in a setup of more than 3~4 clusters
Demai On Fri, Nov 8, 2013 at 1:20 PM, Vladimir Rodionov <[email protected]>wrote: > *I want to setup NxN replication i.e. N clusters each replicating to each > other. N is expected to be around 10.* > > Preparing to thermonuclear war? > > > > > On Fri, Nov 8, 2013 at 1:14 PM, Ishan Chhabra <[email protected] > >wrote: > > > I want to setup NxN replication i.e. N clusters each replicating to each > > other. N is expected to be around 10. > > > > On doing some research, I realize it is possible after HBASE-7709 fix, > but > > it would lead to much more data flowing in the system. eg. > > > > Lets say we have 3 clusters: A,B and C. > > A new write to A will go to B and then C, and also go to C directly via > the > > direct path. This leads to unnecessary network usage and writes to WAL of > > B, that should be avoided. Now imagine this with 10 clusters, it won’t > > scale. > > > > One option is to create a minimum spanning tree joining all the clusters > > and make nodes replicate to their immediate peers in a master-master > > fashion. This is much better than NxN mesh, but still has extra network > and > > WAL usage. It also suffers from a failure scenarios where the a single > > cluster going down will pause replication to clusters downstream. > > > > What I really want is that the ReplicationSource should only forward > > WALEdits with cluster-id same as the local cluster-id. This seems like a > > straight forward patch to put in. > > > > Any thoughts on the suggested approach or alternatives? > > > > -- > > *Ishan Chhabra *| Rocket Scientist | RocketFuel Inc. > > >
