http://zookeeper.apache.org/doc/trunk/zookeeperObservers.html
We can have a DC of observers as stand by and then have a DC of Storm on stand by configured to the local DC. Once the primary DC fails then the Observers in another DC can be changed as Participants and the stand by Storm cluster can deploy the topology. This can be automated if appropriate triggers are available to detect a DC failure. On Fri, Aug 28, 2015 at 11:56 AM Rick Mangi <[email protected]> wrote: > You can, but that could potentially cause a major bottleneck and you would > need at least 6 nodes to handle 1/2 of them dying at once. With zookeeper, > adding more nodes can cause slow performance. I’d love to hear someone tell > me I’m wrong and how to better manage it :) > > I’ve actually been working on writing our storm (non-trident) offsets from > kafka to a separate zookeeper cluster from the one kafka uses to take the > load off of our primary zookeeper cluster. > > > > On Aug 28, 2015, at 1:41 PM, Susheel Kumar Gadalay <[email protected]> > wrote: > > Can't you have zookeeper spawning data centers if there is connectivity. > > Enlarge zookeeper to more than 3 nodes and extra nodes from other data > centers and > also minimum replication to be set. > > > On 8/28/15, Rick Mangi <[email protected]> wrote: > > As long as you are replicating kafka and your offset checkpoints in the > secondary datacenter you should be able to do this. The offsets into kafka > from your spout tell you where you are in your processing (as long as your > topology is acking). > > That said, if those offsets are in zookeeper you will need to find a way to > replicate those to your failover datacenter. > > On Aug 27, 2015, at 3:08 PM, nitin sharma <[email protected]> > wrote: > > hi Andrew, > > our setup includes Kafka --> Storm --> Cassandra .. > > we want to have a backup cluster for Storm, so that in case of any outage > the backup server (topology) picks up from where Active server topology > stopped and process the events and sends them to Cassandra. > > > Regards, > Nitin Kumar Sharma. > > > On Thu, Aug 27, 2015 at 3:04 PM, Andrew Xor <[email protected] > <mailto:[email protected] <[email protected]>>> wrote: > I can try and give you some ideas, but start by first telling us how > you've setup your infrastructure a bit so we can better help you... > > Kindly yours, > > Andrew Grammenos > > -- PGP PKey -- > <https://www.dropbox.com/s/2kcxe59zsi9nrdt/pgpsig.txt> > https://www.dropbox.com/s/yxvycjvlsc111bh/pgpsig.txt > <https://www.dropbox.com/s/ei2nqsen641daei/pgpsig.txt> > > On Thu, Aug 27, 2015 at 9:28 PM, nitin sharma <[email protected] > <mailto:[email protected] <[email protected]>>> wrote: > Hi All, > > I have been asked how can we setup a failover datacenter for Storm, so > that if by any chance (lets say due to power outages) the complete Active > datacenter becomes offline then Standby Datacenter can become alive and > process the real time feeds. > > Does anyone have faced this kind of scenario? what are the best practice > to follow? > > Regards, > Nitin Kumar Sharma. > > >
