There is a configuration setting that you should be submitting to your topology, "topology.workers". You can see how many workers each topology is configured to use by running "storm list" under the "num workers" column. To confirm what Nathan suspects, you might see that this is set to 1 for your topology. You can also see how many workers your topology is configured to use from the Storm UI if you navigate to your running topology and just search the page for "topology.workers".
Changing the number of workers can be done through the configuration that you pass to the topology like (in Java): Config conf = new Config(); conf.setNumWorkers(2); By the way, you can find this info in the Storm tutorial: http://storm.apache.org/documentation/Tutorial.html. There is also some great documentation on understanding parallelism in Storm here: http://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html On Wed, Feb 25, 2015 at 9:39 PM, Vineet Mishra <[email protected]> wrote: > Hi Nathan, > > I guess you missed the first mail of this thread. > > As mentioned before I m having 3 node cluster out of which node 1 is for > nimbus and ui while other 2 nodes are for worker and supervisor. > > The specification of machine is 4 core intel xeon processor which > currently is running with more than 300% cpu utilization. > > At this case how can I Increase number of workers > > Thanks! > On Feb 26, 2015 3:48 AM, "Nathan Leung" <[email protected]> wrote: > >> If you have only one node on the cluster then yes I would recommend to >> add more nodes. If you have more than one I would recommend you increase >> the worker processes in the topology configuration to use the other >> hardware. >> On Feb 25, 2015 10:38 AM, "Vineet Mishra" <[email protected]> wrote: >> >>> If I am taking you right, by adding more workers you mean adding more >>> nodes to the existing cluster or/and enhancing the existing configuration >>> with perhaps 8 or 16 core cpu? >>> >>> On Wed, Feb 25, 2015 at 6:59 PM, Nathan Leung <[email protected]> wrote: >>> >>>> In practice it's very difficult to fully maximize cpu usage, especially >>>> when each thread has to do things like wait for i/o in any way (network, >>>> disk, etc). Since you're cpu limited, or very close to it, you should try >>>> adding more workers. >>>> On Feb 25, 2015 7:48 AM, "Vineet Mishra" <[email protected]> >>>> wrote: >>>> >>>>> Hi Nathan, >>>>> >>>>> You are right, I guess that was the case. >>>>> >>>>> But I am not sure why even after increasing the spouts to 20 I >>>>> couldn't end up with max throughput, I hold 4 core Xeon processor Intel(R) >>>>> Xeon(R) CPU E5-2680 v2 @ 2.80GHz, although I could see the high CPU >>>>> utilization reaching more than 350% but is it the only reason behind the >>>>> less performance gain than expected or do I need to add some configuration >>>>> tuning. >>>>> >>>>> Thanks! >>>>> >>>>> On Wed, Feb 25, 2015 at 5:53 PM, Nathan Leung <[email protected]> >>>>> wrote: >>>>> >>>>>> It looks like you configured your topology to only use one worker >>>>>> process. >>>>>> >>>>>> On Wed, Feb 25, 2015 at 6:39 AM, Vineet Mishra < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> I am running a 3 node storm cluster with a Topology which is >>>>>>> subscribed to Kafka using Kafka-Storm Spout. >>>>>>> >>>>>>> The Cluster is hosting Nimbus on a node and left with 2 other nodes >>>>>>> for the workers and supervisor to function. >>>>>>> >>>>>>> While running the Topology I could see that all the spouts(20) and >>>>>>> bolts(5) tasks are running on the same machine leaving the other node >>>>>>> instance without any CPU utilization. >>>>>>> >>>>>>> [1-1] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 2264180 2264180 >>>>>>> 11.519 2262620 0 [2-2] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 841540 841540 >>>>>>> 19.717 840940 0 [3-3] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 1080080 1080080 >>>>>>> 18.105 1079240 0 [4-4] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 1113900 1113900 >>>>>>> 17.966 1105320 0 [5-5] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 854220 854220 >>>>>>> 19.342 856240 0 [6-6] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 1249640 1249640 >>>>>>> 17.071 1251320 0 [7-7] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 2266520 2266520 >>>>>>> 11.63 2267260 0 [8-8] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 629760 629760 >>>>>>> 20.8 629120 0 [9-9] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 854360 854360 >>>>>>> 19.513 851800 0 [10-10] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 1417380 1417380 >>>>>>> 15.78 1418200 0 [11-11] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 858620 858620 >>>>>>> 20.897 858180 0 [12-12] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 1242980 1242980 >>>>>>> 16.766 1241100 0 [13-13] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 352300 352300 >>>>>>> 19.975 349420 0 [14-14] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 627920 627920 >>>>>>> 20.329 625440 0 [15-15] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 1061880 1061880 >>>>>>> 18.263 1061300 0 [16-16] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 706480 706480 >>>>>>> 20.297 702720 0 [17-17] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 1518800 1518800 >>>>>>> 15.154 1514640 0 [18-18] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 622640 622640 >>>>>>> 20.743 626240 0 [19-19] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 3376960 3376960 >>>>>>> 8.361 3377060 0 [20-20] 31m 7s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 1044940 1044940 >>>>>>> 18.569 1043760 0 >>>>>>> >>>>>>> Same is the case for bolts, >>>>>>> >>>>>>> [22-22]} 32m 13s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 0 0 0.002 0.05 >>>>>>> 1376560 0.042 1376540 0 [23-23]} 32m 13s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 0 0 0.001 0.021 >>>>>>> 347180 0.02 347180 0 [24-24]} 32m 13s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 0 0 0 0.023 >>>>>>> 34600 0.023 34600 0 [25-25]} 32m 13s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 0 0 0 0.017 8380 >>>>>>> 0.014 8380 0 [26-26]} 32m 13s ip-20-0-0-75 6703 >>>>>>> <http://ip-20-0-0-75:8000/log?file=worker-6703.log> 0 0 0.082 0.039 >>>>>>> 22217940 0.041 22217920 0 >>>>>>> >>>>>>> ip-20-0-0-75 is only utilized while their is another host which is >>>>>>> not utilized at all. Please find the supervisor summary below >>>>>>> >>>>>>> 8e550ab9-9e0d-48a7-8899-0963d5f00a41 ip-20-0-0-78 1d 5h 7m 32s 4 0 >>>>>>> 9d98d0b4-1bb4-42e9-9a72-a67b82c64b2c ip-20-0-0-75 5d 23h 57m 53s 4 1 >>>>>>> This is not the regular behavior of Storm as I have seen both the >>>>>>> nodes operating while running the jobs, but how we can comment on the >>>>>>> stability of the storm with this kind off behavior. >>>>>>> >>>>>>> Urgent Issue, Request for expert intervention. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>> >>>>>> >>>>> >>>
