Hi Balazs,
Tuples are acked only when the window slides and the events fall out of the window. So max.spout.pending should be more than max number of tuples in window length + sliding interval. Thanks, Arun On 8/24/16, 8:33 PM, "Balázs Kossovics" <balazs.kossov...@s4m.io> wrote: >Hello, > >I've recently hit a problem with my topology using the windowing mode >when I set the TOPOLOGY_MAX_SPOUT_PENDING too low. > >I have a window lenght of 60 seconds and sliding interval of 1 seconds >with field based timestamping. In my first tests I didn't specify >TOPOLOGY_MAX_SPOUT_PENDING, so when I was working on historical data my >topology died with an OutOfMemoryError because of the practically >unbounded dataflow. Nothing surprising here. > >Just to make sure to avoid the OOMError at the next run, I set >TOPOLOGY_MAX_SPOUT_PENDING to 10, which basically pushed my topology in >a deadlock, because the first 10 tuple was all in the same second, so >the WatermarkTimeEvictionPolicy didn't trigger Action. EXPIRE. Following >this train of thought, imagine that the number of messages during given >window attains this limit: your spout stops emitting, so the tuple which >would make the expire event trigger would never come. > >Am I right in thinking that TOPOLOGY_MAX_SPOUT_PENDING must always be >greate than the number of messages during any timeperiod equivalent to >the window length? > >Best regards, >Balazs >
