Is the watermark advancing past the end of the window? On Mon, Jan 13, 2020 at 2:02 PM Aaron Dixon <atdi...@gmail.com> wrote:
> The window is not empty fwiw; it has elements; I get an early firing pane > for the window but well after the watermark passes there is no ON_TIME > pane. Would this be a bug in Dataflow? Seems fundamental, so I'm concerned > perhaps the Beam spec doesn't obligate ON_TIME firings? > > > > > > On Mon, Jan 13, 2020 at 3:58 PM Luke Cwik <lc...@google.com> wrote: > >> I would have expected an empty on time pane since the default on time >> behavior is FIRE_ALWAYS. >> >> On Mon, Jan 13, 2020 at 1:54 PM Aaron Dixon <atdi...@gmail.com> wrote: >> >>> Can anyone confirm? >>> >>> This is intermittent. Some (it seems, sparse) windows don't get an >>> ON_TIME firing after watermark. Is this a bug or is there a reason to not >>> expect ON_TIME firings for every window? >>> >>> On Mon, Jan 13, 2020 at 3:47 PM Rui Wang <ruw...@google.com> wrote: >>> >>>> If it indeed happened as you have described, I will be very interested >>>> in the expected behaviour. >>>> >>>> Something I remembered before: the trigger condition meets just gives >>>> the runner/engine "permission" to fire, but runner/engine may not fire >>>> immediately. But I don't know if the engine/runner will guarantee to fire. >>>> >>>> >>>> >>>> -Rui >>>> >>>> On Mon, Jan 13, 2020 at 1:43 PM Aaron Dixon <atdi...@gmail.com> wrote: >>>> >>>>> I have the following trigger: >>>>> >>>>> .apply(Window >>>>> .configure() >>>>> .triggering(AfterWatermark >>>>> .pastEndOfWindow() >>>>> .withEarlyFirings(AfterPane >>>>> .elementCountAtLeast(1))) >>>>> .accumulatingFiredPanes() >>>>> .withAllowedLateness(Duration.ZERO) >>>>> >>>>> But in Dataflow I notice that I never get an ON_TIME firing for my >>>>> window -- I only see early firing for elements, and then nothing. >>>>> >>>>> My assumption is that AfterWatermark should give me a last, on-time >>>>> pane under this configuration when the watermark surpasses the window's >>>>> end. >>>>> >>>>> Is my expectation correct? >>>>> >>>>
