Agreed, it adds more moving pieces and hence the added complexity. Would be a nice feature to add.
-Samit On Sat, Oct 18, 2014 at 7:18 PM, Klausen Schaefersinho < [email protected]> wrote: > Hi, > > thanks for that hint. However, I think its a little to complicated and ads > extra complexity to my setup. The external queue would consume extra > performance as it adds some extra "friction". > > Maybe this would be a nice feature for future releases that a topology can > consume input from another topology be default? > > Best Regards, > > Klaus > > > > On Fri, Oct 17, 2014 at 5:41 PM, Samit Sasan <[email protected]> wrote: > >> Best way to do this would be to have you first topology publish on a >> kafka topic and have the next topology read from it that topic via a kafka >> spout. >> >> Hope it helps. >> >> -Samit >> >> On Fri, Oct 17, 2014 at 7:32 PM, Klausen Schaefersinho < >> [email protected]> wrote: >> >>> Hi, >>> >>> I guess in the end I would have still the problem that multiple thread >>> want to read from the fill. Also how do a share an object between >>> different spouts? From my understanding the spouts get serialized / >>> deserialized during deployment and multiple copies of the same object will >>> be created, in case of parallelism. This would also mean there would be >>> multiple file readers... >>> >>> >>> Cheers, >>> >>> Klaus >>> >>> On Fri, Oct 17, 2014 at 3:52 PM, Strulovitch, Zack < >>> [email protected]> wrote: >>> >>>> Why can't you just connect your files reader spout to multiple bolts >>>> (the first bolt of each topology)? >>>> >>>> ------------------------------ >>>> *From:* Klausen Schaefersinho [[email protected]] >>>> *Sent:* Friday, October 17, 2014 3:24 AM >>>> *To:* [email protected] >>>> *Subject:* Read from another Topology >>>> >>>> Hi, >>>> >>>> In my storm setup data arrives in form of files that I have to read >>>> and emit in my spout. Also my topology is very dynamic. Some topologies run >>>> quite long, whereas other can turned on and off frequently. In order to >>>> avoid that I have n spouts reading from the files, I was wondering if could >>>> have just one topology in the cluster which reads from the file and just >>>> emits tuples? All other topologies would than register and that "listen" to >>>> taht topology. >>>> >>>> Cheers, >>>> >>>> Klaus >>>> >>>> ------------------------------ >>>> >>>> This e-mail contains privileged and confidential information intended >>>> for the use of the addressees named above. If you are not the intended >>>> recipient of this e-mail, you are hereby notified that you must not >>>> disseminate, copy or take any action in respect of any information >>>> contained in it. If you have received this e-mail in error, please notify >>>> the sender immediately by e-mail and immediately destroy this e-mail and >>>> its attachments. >>>> >>> >>> >> >
