Hi Spico, The Scheduler mechanism assigns tasks to available worker slots, but does not distribute the workload in real time continuously. Therefore the scheduler approach is unrelated to the network latency your topology may encounter.
An example case for implementing a custom scheduler for network issues could be when a node with a faster connection than the rest contacts a 3rd party API and therefore makes sense to assign all tasks that make these calls to that node. Hope that helps. Regards, Michael On Fri, Sep 5, 2014 at 9:09 AM, Spico Florin <[email protected]> wrote: > Hello! > I have a test case that involves a dataflow containing a spout and N > serial bolts. For better understanding the dataflow looks like this: > Spout->B_1->B_2->...->B_N > where the "->" repersents the transition of the data from one bolt to > another. > Given the fact that by default Storm is using the EvenScheduler algorithm > to distribute the workload between the nodes, you'll encounter the network > latency issue. > In this particular test case, what will be your advice to use as a > Scheduler? To implement one by myself (custom scheduler) or check different > approaches? > > I look forward for your suggestions/ideas. > Best regards, > Florin > -- Michael Vogiatzis Twitter: @mvogiatzis <https://twitter.com/mvogiatzis> http://micvog.com/
