Turned out to not even be relevant to the problem we were having. It was a bolt that hung periodically on processing a single tuple out of several thousand tuples. When the bolt was removed, the processing flowed just fine.
On Wed, Dec 10, 2014 at 1:58 PM, P. Taylor Goetz <[email protected]> wrote: > > Hi Mike, > > Have you set a value for topology.max.spout.pending? If so what value are > you using? > > For a core storm (spouts/bolts) topology that controls how many tuples can > be “in-flight.” For trident it controls how many batches can be outstanding. > > -Taylor > > On Dec 3, 2014, at 2:02 PM, Mike Thomsen <[email protected]> wrote: > > > I've found that our topology will fetch a large block of messages from > kafka and wait about 14-15 minutes before going back to kafka for more > messages. One suggestion I found on Stackoverflow suggested to increase > topology.spout.max.batch.size. Is this the right approach or is there > likely a different reason for the 14-15 minute wait? > > > > Thanks, > > > > Mike > >
