Being very new to storm I'm not sure what to expect in some regards. Ive been playing about with the number of workers/executors/tasks trying to improve throughput on my cluster. I have a 3 nodes, two 4 core and a 2 core node (I can't increase the 3rd node to a medium until the customer gets more cores..). There is a spout that reads from a message bus and a bolt that filter out all but the messages we are interested in processing downstream in the topology. Most messages are filtered out.
I'm assuming that these two components require the most resources as they should be reading/filtering messages at a constant rate, there are two further bolts that are invoked intermittently and hence require less. Ive set the number of workers to 12 (in fact I've noticed it rarely seems to make much difference if I set this to 3/6/9 or 12, there is marginal improvement the higher the value). The spout and filtering bolt I've tried a number of values for in the parallelism hint (I started with 1/4/8/16/20/30/40/60/128/256… ) and I can barely get the throughput to exceed 3500 messages per minute. And the larger hints just grind the system to a halt. Currently I have them both set to 20. The strange thing is that no matter what I do the CPU load is very tiny and is typically 80-90% idle. Suggesting that the topology isn't doing that much work. And tbh I've no idea why this is. Can anyone offer any suggestions? If I understand how this should work, given the number of cores I would think the total number of executors should total 10? Spawning 1 thread per node, I can then set a number of tasks to say 2/4/8 per thread (I've no idea which would be most efficient..). Ive tried something along these lines and my throughput was significantly less, approx. 2000 messages per minute. In fact parallelism hint of 20/24/30 seem to have produced the most throughput. All help/suggestions gratefully received! Thanks David
