[
https://issues.apache.org/jira/browse/BEAM-7785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894143#comment-16894143
]
Jan Lukavský commented on BEAM-7785:
------------------------------------
Although, I might have understood it wrong from the beginning. My observation
was that input watermark can change within single {{onTimer}} call. That made
me think, that it is the synchronization of the watermark and the bundle
processing that needs to be fixed, but - thinking about it more deeply - the
root cause is probably a race between bundle processing on one side, and timer
extraction ({{extractFiredTimers}}) on the other side. That is to say - if
timer is extracted, then *all* timers being processed after that must be aware
of that, otherwise they might setup different timer for time that is less than
the one being extracted. Does that make any sense? :)
> DirectRunner: watermarks are updated asynchronously from bundle processing
> --------------------------------------------------------------------------
>
> Key: BEAM-7785
> URL: https://issues.apache.org/jira/browse/BEAM-7785
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
> Affects Versions: 2.13.0
> Reporter: Jan Lukavský
> Assignee: Jan Lukavský
> Priority: Major
> Time Spent: 4h
> Remaining Estimate: 0h
>
> Watermarks are updated in QuiescenceDriver (by calling fireTimers, which
> calls forceRefresh()) on WatermarkManager. This results in creating timer
> bundles, that are then processed asynchronously as DirectTransformExecutor.
> Because of that, watermarks (input watermarks mostly) might be updated while
> bundle is being processed. That violates assumption, that bundle processing
> should be atomical (with identical external conditions during processing of
> whole bundle).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
