topology.executor.send.buffer.size is the capacity of the send queue. Messages are moved around when the transfer thread is ready. It doesn't wait for a specific number of messages to be available.
If the bolts are always slower than the spout, then increasing the send queue size isn't going to help much. On Thu, Dec 24, 2015 at 11:31 AM, Anishek Agarwal <[email protected]> wrote: > Hello, > > I would like to understand how the transfer of tuples emitted by the > executors move to the workers transfer queue. From what i understand > > > topology.executor.send.buffer.size :1024 > is the number of tuples, once in the executors send queue, will be > transferred to the workers send queue as a batch. > > topology.transfer.buffer.size :1024 > is the number of messages sent by the worker over the network in batch. > > question is: if i have bolts which take a lot of time to execute, then the > number of messages each executor can process is very low per second, say > about 2 tuples per sec, then would the executor outgoing transfer thread > move messages to worker transfer queue after 1024/2 = 512 seconds ? i have > 40 executors per worker. > > if not when does it decide to move messages from executor to worker > transfer queue ? > > storm :0.9.4 > > Thanks > Anishek > -- Regards, Abhishek Agarwal
