Thank you Taylor. It is very clear to me now. On Sat, May 13, 2017 at 5:24 PM, P. Taylor Goetz <ptgo...@gmail.com> wrote:
> Each executor is a separate JVM thread, so in some cases you may want to > conserve threads. Consider a case where you have many tasks, some of which > are very light weight. In that case, it may make sense to group the > lightweight tasks under one executor/thread. JVM threads come with a > certain amount of overhead. > > It's largely a tuning situation. In many cases the defaults are fine. But > we expose it to developers for the situations where the defaults might not > be the best. > > The fact that flux doesn't support it is a simple oversight -- core > functionality was the initial focus. I'll look into adding it to flux, but > would also support any pull request that added the capability. > > (Adding dev@ to include anyone who might be interested.) > > -Taylor > > On May 13, 2017, at 7:24 PM, S G <sg.online.em...@gmail.com> wrote: > > Thanks Anshu. > If you can provide an example, that would be much appreciated. > > On Sat, May 13, 2017 at 1:24 PM, anshu shukla <anshushuk...@gmail.com> > wrote: > >> The main reason behind task in an executor is dynamic scaling based on >> input rate/ resource req. Using storm rebalance. >> >> On Sun, May 14, 2017 at 12:52 AM, S G <sg.online.em...@gmail.com> wrote: >> >>> >>> Hi, >>> >>> As per this guide: http://storm.apache.org/releases/current/Understandin >>> g-the-parallelism-of-a-Storm-topology.html, >>> >>> An executor is a thread containing many tasks. >>> And each task is an instance of a spout or a bolt. >>> >>> I cannot imagine when would someone want to have multiple tasks in an >>> executor. >>> AFAIK, Only use-case possible is when the tasks in an executor have to >>> be synchronous i.e. first task must wait for the second task (in the same >>> executor) to complete and that must wait for the third task (in the same >>> executor) to complete and so on. >>> >>> But that requires the tasks in an executor to be different from each >>> other. >>> Where as the current API does not permit tasks of different type within >>> an executor. >>> >>> So I am "guessing" that multiple tasks in an executor is just a >>> historical artifact with no real use-case. >>> And that is also a reason why Flux has no option to specify num-tasks. >>> >>> $ grep -ri task storm/flux/flux-core/src >>> ===> nothing ! >>> >>> Please confirm if that is a correct understanding. >>> Also, if that is correct, we might be able to squeeze some performance >>> by allowing only a single task per executor. >>> >>> Thanks >>> SG >>> >>> >>> >> >> >> -- >> Thanks & Regards, >> Anshu Shukla >> > >
