The main reason behind task in an executor is dynamic scaling based on input rate/ resource req. Using storm rebalance.
On Sun, May 14, 2017 at 12:52 AM, S G <[email protected]> wrote: > > Hi, > > As per this guide: http://storm.apache.org/releases/current/ > Understanding-the-parallelism-of-a-Storm-topology.html, > > An executor is a thread containing many tasks. > And each task is an instance of a spout or a bolt. > > I cannot imagine when would someone want to have multiple tasks in an > executor. > AFAIK, Only use-case possible is when the tasks in an executor have to be > synchronous i.e. first task must wait for the second task (in the same > executor) to complete and that must wait for the third task (in the same > executor) to complete and so on. > > But that requires the tasks in an executor to be different from each other. > Where as the current API does not permit tasks of different type within an > executor. > > So I am "guessing" that multiple tasks in an executor is just a historical > artifact with no real use-case. > And that is also a reason why Flux has no option to specify num-tasks. > > $ grep -ri task storm/flux/flux-core/src > ===> nothing ! > > Please confirm if that is a correct understanding. > Also, if that is correct, we might be able to squeeze some performance by > allowing only a single task per executor. > > Thanks > SG > > > -- Thanks & Regards, Anshu Shukla
