[
https://issues.apache.org/jira/browse/BEAM-8581?focusedWorklogId=342191&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-342191
]
ASF GitHub Bot logged work on BEAM-8581:
----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Nov/19 21:38
Start Date: 12/Nov/19 21:38
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #10035: [BEAM-8581]
and [BEAM-8582] watermark and trigger fixes
URL: https://github.com/apache/beam/pull/10035#discussion_r345458234
##########
File path: sdks/python/apache_beam/transforms/trigger.py
##########
@@ -1218,7 +1260,9 @@ def _output(self, window, finished, state, watermark,
maybe_ontime):
if timestamp is None:
# If no watermark hold was set, output at end of window.
timestamp = window.max_timestamp()
- else:
+ elif watermark >= window.end or not self.trigger_fn.has_ontime_pane():
+ # Hold the watermark in case there is an empty pane that needs to be
fired
Review comment:
This comment says to hold the watermark, but the code clears it. Reword? Or
maybe negate and have the body be `pass` along with this comment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 342191)
Time Spent: 1h 50m (was: 1h 40m)
> Python SDK labels ontime empty panes as late
> --------------------------------------------
>
> Key: BEAM-8581
> URL: https://issues.apache.org/jira/browse/BEAM-8581
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Sam Rohde
> Assignee: Sam Rohde
> Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The GeneralTriggerDriver does not put watermark holds on timers, leading to
> the ontime empty pane being considered late data.
> Fix: Add a new notion of whether a trigger has an ontime pane. If it does,
> then set a watermark hold to end of window - 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
