Hi, my previous message might be a bit hard to parse for people that are not very deep into the Trigger implementation. So I’ll try to give a bit more explanation right in the mail.
The basic idea is that we observed some basic problems that keep coming up for people on the mailing lists and I want to try and address them. The first problem is with the Trigger semantics and the confusion between triggers that purge the window contents and those that don’t. (For example, using a ContinuousEventTimeTrigger with EventTimeWindows assigner is a bad idea because state will be kept indefinitely.) While working on this we should also tacke the issue of providing composite triggers such as Repeatedly (fires a child-trigger repeatedly), Any (fires when any child trigger fires) and All (fires when all child triggers fire). Lateness. Right now, it is possible to write custom triggers that can deal with late elements and can even behave differently based on the amount of lateness. There is, however, no API for dealing with lateness. We should address this. The third issue is Trigger testability. We should introduce a testing harness for triggers and move the processing time triggers to use a clock provider instead of directly using System.currentTimeMillis(). This will allow testing them deterministically. All of these are expanded upon in the document I linked to before: https://docs.google.com/document/d/1Xp-YBf87vLTduYSivgqWVEMjYUmkA-hyb4muX3KRl08/edit?usp=sharing I think all of this is very important for people working on event-time based pipelines. Feedback is very welcome and I hope that we can expand the document together and come up with good solutions. Cheers, Aljoscha > On 21 Mar 2016, at 17:46, Aljoscha Krettek <aljos...@apache.org> wrote: > > Hi, > I’m also sending this to @user because the Trigger API concerns users > directly. > > There are some things in the Trigger API that I think require some > improvements. The issues are trigger testability, fire semantics and > composite triggers and lateness. I started a document to keep track of things > (https://docs.google.com/document/d/1Xp-YBf87vLTduYSivgqWVEMjYUmkA-hyb4muX3KRl08/edit?usp=sharing). > Please read it if you are interested and want to get involved in this. We’ll > evolve the document together and come up with Jira issues for the subtasks. > > Cheers, > Aljoscha
