Oh nice—that will be great—will look forward to this one! Any idea of
Dataflow will support?
On Sat, Jan 11, 2020 at 9:07 PM Reuven Lax wrote:
> There is now (as of last week) a way to hold back the watermark with the
> state API (though not yet in a released version of Beam). If you set a
>
There is now (as of last week) a way to hold back the watermark with the
state API (though not yet in a released version of Beam). If you set a
timer using withOutputTimetstamp(t), the watermark will be held to t.
On Sat, Jan 11, 2020 at 4:15 PM Aaron Dixon wrote:
> Hi Reuven thanks for your
Hi Reuven thanks for your quick reply
I've tried that but the drag it puts on the watermark was too intrusive.
For example, -- even if just a single user among many decided to remain
logged-in for a few days then the watermark holds everything else back.
This was when using a custom session
Have you looked at using withTimestampCombiner(TimestampCombiner.EARLIEST)?
This will hold the downstream watermark back to the beginning of the window
(presumably the timestamp of the LOGIN event), so you can .call
outputWithTimestamp using the CLICK GREEN timestamp without needing to set
the
I have opened up a PR for BEAM-9008. I wasn't sure if I should initiate
any 'checks' from CI on the PR, so please let me know if I need to and any
other changes/issues. Thanks.
On Fri, Dec 20, 2019 at 7:20 AM Ismaël Mejía wrote:
> For ref this is the JIRA ticket
>
I've just built a pipeline in Beam and after exploring several options for
my use case, I've ended up relying on the deprecated
.outputWithTimestamp() + DoFn#getAllowedTimestampSkew in what seems to me a
quite valid use case. So I suppose this is a vote for un-deprecating this
API (or a teachable
For future reference (for anyone searching mailing list for this or similar
issue), the ticket Kenneth pointed to (
https://issues.apache.org/jira/browse/BEAM-654) is precisely the use case I
have -- "session windows with a terminal/stop event" For now I've opted to
use the State API (
