with the default netty settings storm.messaging.netty.client_worker_threads =1 and storm.messaging.netty.server_worker_threads=1 stopology.max.spout.pending=600 i found all spout have many byte[] object in jvm and the complete latency is very high in spout---about 12000ms, the whole throughput is very low, so i guess there is a bottleneck between workers.
then i increase the thread both of netty client and netty server, storm.messaging.netty.client_worker_threads =20 and storm.messaging.netty.server_worker_threads=20, the whole throughput incease 4* then before, and the complete latency is also decrease to 4000ms. in my topology, 40 spouts( there have 40 partitions), 6 workers, 1 acker. is there netty is a bottleneck? and how to set netty threads?
