Yes, a built-in mechanism is planned in future releases. You can also drop it using a filter for now but the stateful operators will still keep state for old windows.
Matei > On Aug 6, 2016, at 9:40 AM, Amit Sela <[email protected]> wrote: > > I've noticed that when using Structured Streaming with event-time windows > (fixed/sliding), all windows are retained. This is clearly how "late" data is > handled, but I was wondering if there is some pruning mechanism that I might > have missed ? or is this planned in future releases ? > > Thanks, > Amit --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
