Hi Patrick, Sorry I think I confused you with my setup. I have separate nimbus for each storm cluster.
Setup I used for testing, 3 node zk cluster with one node in 3 availability zones. storm cluster 1=nimbus and 3 supervisor in availability zone 1 storm cluster 2=nimbus and 3 supervisor in availability zone 2 Darsh On Thu, Apr 28, 2016 at 10:10 PM, Patrick.Brinton < [email protected]> wrote: > Darsh, > I am in a bit of a crush for a deployment and perf testing but I will ask > my experts to take a look tomorrow. I will set up a single nimbus and 3 > zookeepers in one data center, then I will distribute the supervisors and > see what happens. I think we will always be limited in how we back up > nimbus but I think we should be able to share processing for supervisors > across data centers. > > Keep in touch and let me know if this is really what you are looking to > test. I have a lot of toys at the moment and this seems like a worth while > test for everyone. > > Patrick > > Patrick Brinton Sr. Engineer SWLM | ¤Target | 7000 Target Parkway North > | Brooklyn Park, MN 55445 | 612.599.6523* (ph) *| > [email protected] <[email protected]> > > From: Darsh <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Thursday, April 28, 2016 at 5:29 PM > To: "[email protected]" <[email protected]> > Subject: Re: Distribute load across multiple storm clusters. > Patrick, > > Thank you for replying. I did try but load isn't distributed. Both > clusters are processing all the events individually on the topic. > > > Darsh > > On Thu, Apr 28, 2016 at 2:56 PM, Patrick.Brinton < > [email protected]> wrote: > >> Darsh, >> I have never tried but I have a setup where I could try. As long as you >> point to the same zooKeeper I think it would work. Give it a try and let >> me know if you hit issues. If you do I will mimic your setup and we should >> be able to figure it out. >> >> Patrick >> >> Patrick Brinton Sr. Engineer SWLM | ¤Target | 7000 Target Parkway North >> | Brooklyn Park, MN 55445 | 612.599.6523* (ph) *| >> [email protected] <[email protected]> >> >> From: Darsh <[email protected]> >> Reply-To: "[email protected]" <[email protected]> >> Date: Thursday, April 28, 2016 at 12:39 PM >> To: "[email protected]" <[email protected]> >> Subject: Distribute load across multiple storm clusters. >> >> Hi, >> >> >> Is it possible to to distribute load across 2 clusters if I deploy same >> topology to 2 storm clusters with same spout id? >> >> >> We have 2 separate storm clusters with 0.10.0 version of storm running in >> 2 different availability zones. We are using storm-kafka spout (with simple >> consumer) to process data from kafka and external common zookeeper to store >> the kafka offsets.Kafka Topic has 32 partitions. 8 executors( >> *parallelism_hint*) for spout in each cluster. >> >> >> >> >> >> >> Thanks >> >> Darsh >> > > > > -- > Thanks > > Darsh > -- Thanks Darsh
