The best tool for topology tuning is Storm UI. Here you can learn your process latency and capacity of your storm components. Start with some baseline, for example, having all parallelism setting to 1. Then gradually increase parallelism until you see capacity numbers well below 1.0. In general, try to increase number of executors first, before increasing number of workers. When increasing number of executors (threads) does not give any improvements, then start adding additional workers. Also, if your components more IO bound then increasing number of task helps as well.
In addition to Storm UI,and resources Francesco pointed out, you can use storm metrics API to send data to graphite. http://www.michael-noll.com/blog/2013/11/06/sending-metrics-from-storm-to-graphite/ Upcoming book, Storm Applied <http://www.manning.com/sallen/> has a chapter about tuning Storm (chapter 6). Thanks Eugene Dvorkin On Sat, Jul 5, 2014 at 7:19 AM, <[email protected]> wrote: > i ' ve found some good advices reading these articles > > > http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/ > > > http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/ > > > > > > > > > Il 2014-07-05 12:12 唐思成 ha scritto: > > everyone, do you have some best practice or pattern for tunning the storm > cluster, like how to set the number of workers, the numbers of spout and > bolts, etc.. > Any advice is welcome. > > 2014-07-05 > ------------------------------ > 唐思成 > > > > -- Eugene Dvorkin Software Engineer New York City Storm User Group - organizer WebMD email: [email protected] phone: 646-250-9649 eugenedvorkin.com Connect with me on: LinkedIn <http://www.linkedin.com/in/eugenedvorkin> Twitter <http://twitter.com/edvorkin>
