Another alternative is to be able to set special timers, e.g. end of window and expiration of window. That at least addresses (2).
On Tue, Mar 28, 2017 at 1:27 PM, Kenneth Knowles <k...@google.com.invalid> wrote: > Hi all, > > I have a little extension to the stateful DoFn annotations to circulate for > feedback: Allow a method to be annotated with @OnWindowExpiration to > automatically get a callback at some point after the window has expired, > but before the state for the window has been cleared. > > Today, a user can pretty easily get the same effect by setting a timer for > the end of the window + allowed lateness in their @ProcessElement calls. > But having just one annotation for it has a couple nice benefits: > > 1. Some users assume a naive implementation so they are concerned that > setting a timer repeatedly is costly. This eliminates the cause for user > alarm and allows a runner to do a better job in case it didn't already do > it efficiently. > > 2. Getting the allowed lateness to be available to your @ProcessElement is > a little crufty. > > 3. Often, if you don't have @OnWindowExpiration, you are leaving behind > state that might contain data that is otherwise lost. So I would even > consider making it mandatory (with some way of indicating state you don't > care about dropping) though that could be annoying. > > Another interesting moment in a window's lifecycle is @EndOfWindow. This is > not critical for correctness, though. > > Thoughts? > > Kenn >
