To what number / metric are you referring when you say, "When number of tuples increases in queue"? What you are describing sounds like the beginning of queue explosion. If so, increasing max spout pending will make the situation worse.
Thank you for your time! +++++++++++++++++++++ Jeff Maass <[email protected]> linkedin.com/in/jeffmaass stackoverflow.com/users/373418/maassql +++++++++++++++++++++ On Tue, May 12, 2015 at 6:22 AM, Kutlu Araslı <[email protected]> wrote: > Hi everyone, > > Our topology consumes tuples from a Kestrel MQ and runs a series of bolts > to process items including some db connections. Storm version is 0.8.3 and > supervisors are run on VMs. > When number of tuples increases in queue, we observe that, a single tuple > execution time also rise dramatically in paralel which ends up with a > throttle behaviour. > In the meantime CPU and memory usage looks comfortable.From database > point, we have not observed a problem so far under stress. > Is there any configuration trick or an advice for handling such a load? > There is already a limit on MAX_SPOUT_PENDING as 32. > > Thanks, > > >
