EvenScheduler doesn't seem to do the trick. I've set the scheduler to be EvenScheduler, but the two topologies we've got are still being assigned to the same supervisor, when there's 3 possible supervisors to assign to. It's hard to tell what exactly EvenScheduler is doing, is there some specification of what exactly EvenScheduler and DefaultScheduler do somewhere?
From: bijoy deb [mailto:[email protected]] Sent: 22 March 2014 05:39 To: [email protected] Subject: Re: Multiple Storm components getting assigned to same worker slot despite of free slots being available Thanks Nathan.So, I believe setting storm.scheduler to EvenScheduler,as suggested by Drew should do the trick? However, I still have one doubt.With reference to my use case, what I was looking for is that each component (spout/bolt instance) should get assigned to different slot,since I have free slots available;but instead multiple instances are getting assigned to same slot. Does setting the scheduler to EvenScheduler ensure even distribution of tasks across slots within a single worker machine also,or does it just ensure even distribution across multiple worker machines only. Also,shouldn't this scheduler property setting be there by default,since we would always like to assign tasks to empty slots first? Thanks Bijoy On Sat, Mar 22, 2014 at 10:57 AM, Nathan Marz <[email protected]<mailto:[email protected]>> wrote: topology.optimize doesn't do anything at the moment. It was something planned for in the early days but turned out to be unecessary. On Fri, Mar 21, 2014 at 9:00 PM, bijoy deb <[email protected]<mailto:[email protected]>> wrote: Thanks Drew.I am going to try those options and see if that helps. Thanks Bijoy On Fri, Mar 21, 2014 at 10:37 PM, Drew Goya <[email protected]<mailto:[email protected]>> wrote: Take a look at "topology.optimize" and "storm.scheduler" I had the same issue and I found that setting topology.optimize to false and storm.scheduler to "backtype.storm.scheduler.EvenScheduler" gave me the even distribution of components I was looking for. On Fri, Mar 21, 2014 at 2:50 AM, bijoy deb <[email protected]<mailto:[email protected]>> wrote: Hi, I am running a topology using Storm (version 0.9.1),on a cluster of 3 nodes (3x4=12 slots). My topology has 1 spout(parallelism=2),bolt A (parallelism=2),bolt B (parallelism=1) and bolt C(parallelism=1).Number of tasks (numTasks) for each component is default(1).Number of workers is set as 5. Given above scenario,when I submit the topology,I can see 5 slots are used up and 7 are free (out of 12).But still one instance of the spout and bolt C are going to the same worker slot (e.g port 6703 of node 1). Shouldn't Storm be ensuring that components are assigned to distinct unused slots as long as there are empty slots available? Or is there some configuration that I have missed or misconfigured here? I have pasted the screenshot of Storm UI below for reference. Thanks Bijoy -- Twitter: @nathanmarz http://nathanmarz.com<http://nathanmarz.com/>
