Hi Millind, I apologize, I need to ask my question more precisely. :) I know that I set the number of executors for storm, but the question is specifically in regards to the KafkaBolt. I am thinking that the max number of KafkaBolt executors should be no more than the number of topics times the number of partitions. In the example I presented in this thread, that would be 2 topics * 10 partitions per topic = 20.
Just wondering if my logic makes sense and/or if there is a better parallelism strategy for KafkaBolts. Thanks --John On Wed, Jan 13, 2016 at 2:44 AM, Milind Vaidya <[email protected]> wrote: > Hi John, > > No. It is not driven by number of topics or partitions. This is a number > you can configure while setting the bolt into the topology builder. > > Here is useful link to clarify more : > http://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html > > > > On Tue, Jan 12, 2016 at 3:00 PM, John Yost <[email protected]> wrote: > >> Hi Everyone, >> >> Is the number of KafkaBolt executors driven by number of topics X number >> of partitions? For example, if I have two topics, each of which has ten >> partitions, does 20 KafkaBolt executors make sense? Just trying to figure >> out the optimal parallelism. >> >> Any insights much appreciated--thanks! >> >> --John >> > >
