I guess this is a question where there r no really correct answers. I'll certainly avoid#1 as it is better to keep logic separate and lightweight.
If your downstream bolts are same, then it makes senses to keep them in same topology but if they r totally different, I'll keep them in two different topologies. That will allow me to independently deploy and scale the topology. But if the rest of logic is same I topology scaling and resource utilization will be better with one topology. I hope this helps.. Sent somehow.... > On Oct 11, 2015, at 9:07 AM, Ankur Garg <[email protected]> wrote: > > Hi , > > So I have a situation where I want to read messages from different queues > hosted in a Rabbitmq Server . > > Now , there are three ways which I can think to leverage Apache Storm here :- > > 1) Use the same Spout (say Spout A) to read messages from different queues > and based on the messages received emit it to different Bolts. > > 2) Use different Spout (Spout A and Spout B and so on) within the same > topology (say Topology A) to read messages from different queues . > > 3) Use Different Spouts one within eachTopology (Topology A , Topology B and > so on) to read messages from different queues . > > Which is the best way to process this considering I want high throughput > (more no of queue messages to be processed concurrently) . > > Also , If In use same Topology for all Spouts (currently though requirement > is for 2 spouts) will failure in one Spout (or its associated Bolts) effect > the second or will they both continue working separately even if some failure > is in Spout B ? > > Cost wise , how much would it be to maintain two different topologies . > > Looking for inputs from members here. > > Thanks > Ankur > >
