Ideally, each time nextTuple is called, you should be emitting only one tuple. Of course, you can emit more than one, but then it would be better to monitor the latency and emit only as many tuples which can be ack'ed within a latency of 2.5 second. Make sure you have enough of workers Increase TOPOLOGY_MESSAGE_TIMEOUT_SECS Increase stormConfig.setNumWorkers(someNumber); and stormConfig.setNumAckers(someNumber); Each storm node will have 4 slots which can handle 4 workers, so create as many workers as you have slots. Slots = number of nodes * 4. If you have more workers than slots, then Storm will have to handle more than one worker on a single slot, which will be a little slower. Having number of workers = number of tasks (number of spouts and bolts) is also helpful to avoid lags.
If you really want to increase the number of emits phenomenally, then use a separate program to put objects into a queue like RabbitMQ or any of the other queue programs available. Then, create multiple spout instances which will read from this queue and emit. This way, you'll have multiple spouts emitting tuples, and you can have multiple bolts which take tuples from these spouts and process the data. On Thu, Nov 24, 2016 at 11:02 AM, Walid Aljoby <[email protected]> wrote: > Hi Navin, > > Yes, I meant by the sending rate; the outgoing tuples from the spout, as > the Representative for data source, to the computation bolts. > The question about tuning the respective parameters for increasing the > spout emitting tuples. Actually, I tried different values for max spout > pending, but not much improvement in the application throughput. Hence, I > asked if other parameters affect the speed of emitting tuples. > > Thank you and Regards, > -- > WA > > > ------------------------------ > *From:* Navin Ipe <[email protected]> > *To:* [email protected]; Walid Aljoby <[email protected]> > *Sent:* Thursday, November 24, 2016 12:54 PM > *Subject:* Re: Storm sending rate > > Please remember that we cannot read your mind. A little more elaboration > on what problem you are facing and what you mean by "sending rate" would > help. > > On Wed, Nov 23, 2016 at 5:56 PM, Walid Aljoby <[email protected]> > wrote: > > Hi everyone, > > Could anyone has an experience to explain the factors affecting sending > rate in Storm? > > > Thank you > -- > Regards > WA > > > > > -- > Regards, > Navin > > > -- Regards, Navin
