So I have a situation where the tuple received on Spout has to be saved to mysql database and mongoDb as well .
What should be better . Using 1 bolt to save it into mysql and MongoDb or 2 seperate Bolts (One for saving into mysql and other for saving into Mongo). What happens when the exception occurs while saving into mysql ? I believe I will get acknowledgement inside the fail method in my Spout . So If I reprocess it using 2 bolts , I believe it will again be sent to Bolt for saving into Mongo database . If the above is true , will having 2 seperate bolts be of any advantage ? how can I configure things so that Failure while inserting into mysql does not impact inserting into MongoDb . Thanks Ankur On Sun, Oct 11, 2015 at 10:57 PM, Ravi Sharma <[email protected]> wrote: > That depends if ur spout error has affected jvm or normal application error > > performance issue in case of lot of errors, I don't think there is any > issue be coz of errors themselves but ofcourse if u r retrying these > messages on failure then that means u will be processing lot of messages > then normal and overall throughput will go down > > Ravi > > If ur topology has enabled acknowledgment that means spout will always > receive > On 11 Oct 2015 18:15, "Ankur Garg" <[email protected]> wrote: > >> >> Thanks for the reply Abhishek and Ravi . >> >> One question though , going with One topology with multiple spouts >> ...What if something goes wrong in One spout or its associated bolts .. >> Does it impact other Spout as well? >> >> Thanks >> Ankur >> >> On Sun, Oct 11, 2015 at 10:21 PM, Ravi Sharma <[email protected]> >> wrote: >> >>> No 100% right ansers , u will have to test and see what will fit.. >>> >>> persoanlly i wud suggest Multiple spouts in one Topology and if you have >>> N node where topology will be running then each Spout(reading from one >>> queue) shud run N times in parallel. >>> >>> if 2 Queues and say 4 Nodes >>> then one topolgy >>> 4 Spouts reading from Queue1 in different nodes >>> 4 spouts reading from Queue2 in different nodes >>> >>> Ravi. >>> >>> On Sun, Oct 11, 2015 at 5:25 PM, Abhishek priya < >>> [email protected]> wrote: >>> >>>> I guess this is a question where there r no really correct answers. >>>> I'll certainly avoid#1 as it is better to keep logic separate and >>>> lightweight. >>>> >>>> If your downstream bolts are same, then it makes senses to keep them in >>>> same topology but if they r totally different, I'll keep them in two >>>> different topologies. That will allow me to independently deploy and scale >>>> the topology. But if the rest of logic is same I topology scaling and >>>> resource utilization will be better with one topology. >>>> >>>> I hope this helps.. >>>> >>>> Sent somehow.... >>>> >>>> > On Oct 11, 2015, at 9:07 AM, Ankur Garg <[email protected]> wrote: >>>> > >>>> > Hi , >>>> > >>>> > So I have a situation where I want to read messages from different >>>> queues hosted in a Rabbitmq Server . >>>> > >>>> > Now , there are three ways which I can think to leverage Apache Storm >>>> here :- >>>> > >>>> > 1) Use the same Spout (say Spout A) to read messages from different >>>> queues and based on the messages received emit it to different Bolts. >>>> > >>>> > 2) Use different Spout (Spout A and Spout B and so on) within the >>>> same topology (say Topology A) to read messages from different queues . >>>> > >>>> > 3) Use Different Spouts one within eachTopology (Topology A , >>>> Topology B and so on) to read messages from different queues . >>>> > >>>> > Which is the best way to process this considering I want high >>>> throughput (more no of queue messages to be processed concurrently) . >>>> > >>>> > Also , If In use same Topology for all Spouts (currently though >>>> requirement is for 2 spouts) will failure in one Spout (or its associated >>>> Bolts) effect the second or will they both continue working separately even >>>> if some failure is in Spout B ? >>>> > >>>> > Cost wise , how much would it be to maintain two different topologies >>>> . >>>> > >>>> > Looking for inputs from members here. >>>> > >>>> > Thanks >>>> > Ankur >>>> > >>>> > >>>> >>> >>> >>
