Prefetch is how many messages each spout takes from rabbitmq at a time. Having it higher than max spout pending can cause undesired behavior.
Is your code available anywhere online? On Sun, Feb 23, 2014 at 12:42 PM, hyakunin <[email protected]> wrote: > I had this constant in the AMQP_Spout, I guess it's something related > with the outstanding non-ack tuples > > private static final long DEFAULT_PREFETCH_COUNT = 100; > > when increased this limit I now have the limit in the maxSpoutPending as > Sean said > > For my needs I'll use rabbitmq and storm without any ack or auto-ack... > > > > > > 2014-02-22 20:59 GMT+01:00 Sean Allen <[email protected]>: > > are you using maxSpoutPending? >> >> >> >> On Thu, Feb 20, 2014 at 3:11 PM, hyakunin <[email protected]> wrote: >> >>> Hi, I have a Storm Topology composed of a AMPQ_Spout and a bolt capable >>> of writing to a MongoDB Collection >>> >>> The problem is that once I run the Java Rabbitmq program that sends >>> tuples to Rabbitmq queue; the Storm topology pulls the Rabbitmq taking 100 >>> tuples and waiting for some minutes before it pulls another bunch of 100 >>> tuples. >>> >>> I have revised all the code and simplified the system as much I could >>> but I still have the same problem >>> >>> does anybody have any clue of what can be happening or at least how to >>> detect where's this weird bottleneck? >>> >>> thanks >>> David >>> >> >> >> >> -- >> >> Ce n'est pas une signature >> > > -- Ce n'est pas une signature
