Hi Shen, This is a documentation issue. The Beam model switched from dropping individual elements to expiring windows. So now elements are droppable if they belong to an expired window. This works a little better with the purpose of windowing and allowed lateness: to say when an aggregation is "complete". Any element that manages to make it to an aggregation before the accumulator is expired is allowed to be included now and only after the whole window expires we drop any further incoming elements for that window.
Kenn On Mon, Jan 22, 2018 at 10:52 AM, Shen Li <[email protected]> wrote: > Hi, > > The Window#withAllowedLateness(Duration) doc says "Any elements that are > later than this as decided by the system-maintained watermark will be > dropped". Can the runner safely discard a tuple that violates the allowed > lateness in the Window operator? Or does it have to drop it in the > downstream GBK operator just in case that there could be another Window > transform in between overriding the allowed lateness (or other > configurations)? > > Thanks, > Shen >
