windowing & max spout pending

[Balázs Kossovics]

Hello,

I've recently hit a problem with my topology using the windowing mode 
when I set the TOPOLOGY_MAX_SPOUT_PENDING too low.

I have a window lenght of 60 seconds and sliding interval of 1 seconds 
with field based timestamping. In my first tests I didn't specify 
TOPOLOGY_MAX_SPOUT_PENDING, so when I was working on historical data my 
topology died with an OutOfMemoryError because of the practically 
unbounded dataflow. Nothing surprising here.

Just to make sure to avoid the OOMError at the next run, I set 
TOPOLOGY_MAX_SPOUT_PENDING to 10, which basically pushed my topology in 
a deadlock, because the first 10 tuple was all in the same second, so 
the WatermarkTimeEvictionPolicy didn't trigger Action. EXPIRE. Following 
this train of thought, imagine that the number of messages during given 
window attains this limit: your spout stops emitting, so the tuple which 
would make the expire event trigger would never come.

Am I right in thinking that TOPOLOGY_MAX_SPOUT_PENDING must always be 
greate than the number of messages during any timeperiod equivalent to 
the window length?

Best regards,
Balazs