Thanks Nathan.So, I believe setting storm.scheduler to EvenScheduler,as suggested by Drew should do the trick?
However, I still have one doubt.With reference to my use case, what I was looking for is that each component (spout/bolt instance) should get assigned to different slot,since I have free slots available;but instead multiple instances are getting assigned to same slot. Does setting the scheduler to EvenScheduler ensure even distribution of tasks across slots within a single worker machine also,or does it just ensure even distribution across multiple worker machines only. Also,shouldn't this scheduler property setting be there by default,since we would always like to assign tasks to empty slots first? Thanks Bijoy On Sat, Mar 22, 2014 at 10:57 AM, Nathan Marz <[email protected]> wrote: > topology.optimize doesn't do anything at the moment. It was something > planned for in the early days but turned out to be unecessary. > > > On Fri, Mar 21, 2014 at 9:00 PM, bijoy deb <[email protected]>wrote: > >> Thanks Drew.I am going to try those options and see if that helps. >> >> Thanks >> Bijoy >> >> >> On Fri, Mar 21, 2014 at 10:37 PM, Drew Goya <[email protected]> wrote: >> >>> Take a look at "topology.optimize" and "storm.scheduler" >>> >>> I had the same issue and I found that setting topology.optimize to false >>> and storm.scheduler to "backtype.storm.scheduler.EvenScheduler" gave me the >>> even distribution of components I was looking for. >>> >>> >>> On Fri, Mar 21, 2014 at 2:50 AM, bijoy deb <[email protected]>wrote: >>> >>>> Hi, >>>> >>>> I am running a topology using Storm (version 0.9.1),on a cluster of 3 >>>> nodes (3x4=12 slots). My topology has 1 spout(parallelism=2),bolt A >>>> (parallelism=2),bolt B (parallelism=1) and bolt C(parallelism=1).Number of >>>> tasks (numTasks) for each component is default(1).Number of workers is set >>>> as 5. >>>> >>>> Given above scenario,when I submit the topology,I can see 5 slots are >>>> used up and 7 are free (out of 12).But still one instance of the spout and >>>> bolt C are going to the same worker slot (e.g port 6703 of node 1). >>>> Shouldn't Storm be ensuring that components are assigned to distinct unused >>>> slots as long as there are empty slots available? Or is there some >>>> configuration that I have missed or misconfigured here? >>>> >>>> I have pasted the screenshot of Storm UI below for reference. >>>> >>>> Thanks >>>> Bijoy >>>> >>>> >>> >> > > > -- > Twitter: @nathanmarz > http://nathanmarz.com >
