And one more question while we're at it: what if you have events happening
every second within the window? Do you really want to emit a new pane every
second as the watermark progresses (assuming it progresses relatively
smoothly)? What if we're talking differences of event times of
milliseconds? Is one pane per millisecond what you want?

-Tyler

On Fri, Dec 2, 2016 at 10:41 AM Kenneth Knowles <k...@google.com> wrote:

> Thanks for laying out some details.
>
> On Thu, Dec 1, 2016 at 7:09 PM, Manu Zhang <owenzhang1...@gmail.com>
> wrote:
>
> Yes, the difficulty is to define that trigger. The existing triggers fire
> at the end of window. (I could be mistaken, which will be good news)
>
>
> You are not mistaken that the only existing event time trigger is the one
> that fires at the end of the window. The trigger you describe would be a
> new primitive trigger. It fits with the design, if we ensure monotonicity,
> etc. Actually implementing it in the backend is easy, of course. We
> actually had something like it, but didn't quite nail it down so we removed
> it until we had a solid use case and design for it.
>
> B and C which are not mutually exclusive
> More on my use case. Say a user visits http://foo at 1, http://foo/bar at
> 4 and back to http://foo at 5 all in a Session
> we would want to emit
>
> http://foo  when the watermark passes 1
> http://foo -> http://foo/bar when the watermark passes 4
> http://foo -> http://foo/bar -> http://foo when the watermark passes 5
>
>
> What would you want to emit when the watermark jumps from 0 to 7 and all
> three of the above are buffered?
>
> What would you want to emit when the watermark was at 9 and
> http://foo/bizzle came in with timestamp 3?
>
> Kenn
>
>
>
>
> On Fri, Dec 2, 2016 at 10:12 AM Ben Chambers <bchamb...@google.com> wrote:
>
> As a clarifying question:
>
> If you have three elements in the pane with timestamps [1, 4, 5], would
> you:
> A. want to emit that entire pane when the watermark passes 1
> B. want to emit that entire pane when the watermark passes 5
> C. emit a fragment of that pane containing only the first element when the
> watermark passes 1
>
> On Thu, Dec 1, 2016 at 6:01 PM Tyler Akidau <taki...@google.com> wrote:
>
> So what you want is essentially a trigger that fires when the watermark
> has passed the event time of the oldest un-emitted element in the current
> pane? You could them presumably wrap this in a repeat to get the overall
> desired semantics, right?
>
> -Tyler
>
>
> On Fri, Dec 2, 2016 at 7:32 AM Manu Zhang <owenzhang1...@gmail.com> wrote:
>
> My use case is to track user trajectory based on page view event when they
> visit a website.  The input would be like a list of PageView(userId, url,
> eventTimestamp) with watermarks (= eventTimestamp - duration). I'm trying
> Sessions with event time trigger. Note we can't wait for the end of session
> window due to latency requirement. Instead, we want to emit the user
> trajectories whenever a buffered PageView's event time is passed by
> watermark.
>
> On Fri, Dec 2, 2016 at 5:41 AM Lukasz Cwik <lc...@google.com> wrote:
>
> Can you provide more details about the problem your trying to solve with
> some examples showing input and the expected output?
>
>
>
>
> On Wed, Nov 30, 2016 at 11:08 PM, Manu Zhang <owenzhang1...@gmail.com>
> wrote:
>
> Hi,
>
> Recently I’m addressing a problem where users want to trigger after
> watermark past each element (i.e. in the middle of event-time window). I
> fail to find an existing trigger that does so. Any idea on model this
> problem with Beam ?
>
> Thanks,
> Manu Zhang
>
>
>

Reply via email to