[
https://issues.apache.org/jira/browse/BEAM-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15282023#comment-15282023
]
Kenneth Knowles commented on BEAM-279:
--------------------------------------
Your final sentence is important. The best way to implement bounded sessions
today is probably with a special WindowFn.
More broadly, the idea of a bounded sessions is compatible with event time
processing, but neither the trigger-based approach nor the WindowFn based
approach actually express it. So I don't think we need to work too hard to make
either one perfect (but both should work heuristically, as you say).
Here is an approach that I think respects event time processing: the full
unbounded session is buffered, with the ability to emit & discard bounded
prefixes as they expire (thus no further data could alter it). The latency of
output will be poor if allowed lateness is high. But, in fact, it is also OK to
emit the ON_TIME pane for a bounded prefix; further late data merging in with
it would just make the session's EOW earlier, and triggering sooner, which is
allowed by monotonicity.
> Make sure we unit test bounded sessions
> ---------------------------------------
>
> Key: BEAM-279
> URL: https://issues.apache.org/jira/browse/BEAM-279
> Project: Beam
> Issue Type: Bug
> Reporter: Mark Shields
>
> A few customers have been using Window.into(Sessions...) and of course
> quickly realize they are exposed to unbounded sessions.
> We should have unit tests to confirm various combinations of
> AfterPane.elementCountAtLeast and AfterProcessingTime... correctly force
> sessions to be broken apart.
> We should also check this all works with repeated messages with the same
> timestamp (since they will create the exact same session window and can thus
> see trigger state from previous sessions).
> At some point we may may flow into reworking bounded sessions to be done
> directly rather than via Sessions plus triggers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
