Hello! Thank you for your responses. My concerns are related with the fact that the bolts will be placed in a cluster in a round robin fashion. Thus, you'll encounter the latency due to the network transfer between nodes. In my opinion, in my case the more I collocate the bolts on the same node (and also slot) the more I'll decrease the network latency and thus the time. I'm I right with my assumptions? Regards, Florin
On Fri, Sep 5, 2014 at 1:36 PM, Vikas Agarwal <[email protected]> wrote: > There are schedulers which can adapt themselves based on topology workload > and historical performances. > > I found a good paper on this: > http://www.dis.uniroma1.it/~midlab/articoli/ABQ13storm.pdf > > > On Fri, Sep 5, 2014 at 3:20 PM, Michael Vogiatzis < > [email protected]> wrote: > >> Hi Spico, >> >> The Scheduler mechanism assigns tasks to available worker slots, but does >> not distribute the workload in real time continuously. Therefore the >> scheduler approach is unrelated to the network latency your topology may >> encounter. >> >> An example case for implementing a custom scheduler for network issues >> could be when a node with a faster connection than the rest contacts a 3rd >> party API and therefore makes sense to assign all tasks that make these >> calls to that node. >> >> Hope that helps. >> Regards, >> Michael >> >> >> On Fri, Sep 5, 2014 at 9:09 AM, Spico Florin <[email protected]> >> wrote: >> >>> Hello! >>> I have a test case that involves a dataflow containing a spout and N >>> serial bolts. For better understanding the dataflow looks like this: >>> Spout->B_1->B_2->...->B_N >>> where the "->" repersents the transition of the data from one bolt to >>> another. >>> Given the fact that by default Storm is using the EvenScheduler >>> algorithm to distribute the workload between the nodes, you'll encounter >>> the network latency issue. >>> In this particular test case, what will be your advice to use as a >>> Scheduler? To implement one by myself (custom scheduler) or check different >>> approaches? >>> >>> I look forward for your suggestions/ideas. >>> Best regards, >>> Florin >>> >> >> >> >> -- >> Michael Vogiatzis >> Twitter: @mvogiatzis <https://twitter.com/mvogiatzis> >> http://micvog.com/ >> > > > > -- > Regards, > Vikas Agarwal > 91 – 9928301411 > > InfoObjects, Inc. > Execution Matters > http://www.infoobjects.com > 2041 Mission College Boulevard, #280 > Santa Clara, CA 95054 > +1 (408) 988-2000 Work > +1 (408) 716-2726 Fax > >
