Hi, Are you setting the number of workers to 10? Ideally, Storm should distribute the spouts/bolts equally between the workers, provided you’ve set the number of workers right.
You can set the number of workers as follows: Config conf = new Config(); conf.setNumWorkers(10); You can read this blog-post for more details on Storm’s parallelism - http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/#configuring-the-parallelism-of-a-topology -- Thanks & Regards, Gufran Pathan | +91-9566811502 | www.mu-sigma.com<http://www.mu-sigma.com/> | “Mistakes are the portals of discovery” – James Joyce From: Raman Grover [mailto:[email protected]] Sent: Thursday, June 12, 2014 4:12 PM To: [email protected] Subject: Fixing the Bolts to Specific Hosts on a Storm cluster Hi I have set up a Storm cluster and also a MongoDB cluster using a common set of 10 nodes. My topology involves a spout that connects with a bolt using fields grouping. The bolt contains the logic to put received data into MongoDB. The degree of parallelism for spout is 1 while that for the bolt is set to10. I need to ensure that the bolt runs with one instance on each of the 10 nodes, to involve all machines. However by default, I see that single spout and the 10 bolt instances all run on a single node. The rest 9 nodes are left idle. Can we somehow have Storm to run the 10 bolt instances on individual machines on my cluster rather than scheduling all of them on a single node? Please suggest. Regards, Raman Disclaimer: http://www.mu-sigma.com/disclaimer.html
