That's unfortunate, I was hoping for a built-in way to do it, but that's ok. Thanks for the quick response.
- Arthur On Mon, Nov 7, 2016 at 11:56 AM, P. Taylor Goetz <[email protected]> wrote: > Yes, to do what you want you would need to implement a custom scheduler. > > More details can be found here: http://storm.apache.org/ > releases/1.0.2/Storm-Scheduler.html > > -Taylor > > On Nov 7, 2016, at 11:31 AM, Arthur Maciejewicz <[email protected]> > wrote: > > Hello All, > > Is there a way to make the "shape" of topology parallelism explicit? > For example, assume: > > * W workers on W nodes > * 2 spouts with 1 executor each (total of 2 executors) > * 1 "mapper" bolt with M executors > * 1 "process" bolt with N executors. > > Is it currently possibly to co-locate the spouts with the "mappers" in 2 > workers, while pinning the N "process" executors to the remaining (W-2) > workers? > > Visually this is what I want to do: > > > 2 Workers W-2 Workers > > +-----------+ > +----------+ | process | > | spout | +-----------+ > | + +--> +-----------+ > | map | | process | > +----------+ +-----------+ > +-----------+ > +----------+ | process | > | spout | +-----------+ > | + +--> ... > | map | +-----------+ > +----------+ | process | > +-----------+ > +-----------+ > | process | > +-----------+ > > Thanks, > > Arthur > > >
