[
https://issues.apache.org/jira/browse/BEAM-10676?focusedWorklogId=469563&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-469563
]
ASF GitHub Bot logged work on BEAM-10676:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Aug/20 07:45
Start Date: 12/Aug/20 07:45
Worklog Time Spent: 10m
Work Description: mxm commented on a change in pull request #12531:
URL: https://github.com/apache/beam/pull/12531#discussion_r469068866
##########
File path: sdks/python/apache_beam/runners/worker/bundle_processor.py
##########
@@ -647,7 +647,7 @@ def set(self, ts):
windows=(self._window, ),
clear_bit=False,
fire_timestamp=ts,
- hold_timestamp=self._input_timestamp,
+ hold_timestamp=ts,
Review comment:
Do you mean to keep the input timestamp if we have a processing time
timer?
As far as I'm concerned, the hold/output timestamp is elevant for event time
timers. Processing timers do not operate in event time and thus do not respect
this timestamp, no matter what it is set to. Unless there is a good reason for
introducing this conditional behavior, I'd keep it as-is.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 469563)
Time Spent: 0.5h (was: 20m)
> Timers use the input timestamp as the timer output timestamp which prevents
> watermark progress
> ----------------------------------------------------------------------------------------------
>
> Key: BEAM-10676
> URL: https://issues.apache.org/jira/browse/BEAM-10676
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, sdk-py-harness
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: P2
> Fix For: 2.24.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> By default, the Python SDK adds a timer output timestamp equal to the current
> timestamp of an element. This is problematic because
> 1. We hold back the output watermark on the current element's timestamp for
> every timer
> 2. It doesn't match the behavior in the Java SDK which defaults to using the
> fire timestamp as the timer output timestamp (and adds a hold on it)
> 3. There is no way for the user to influence this behavior because there is
> no user-facing API
> https://github.com/apache/beam/blob/dfadde2d3ee0a0487362dbcca80388fdc2ef2302/sdks/python/apache_beam/runners/worker/bundle_processor.py#L650
> We should use the fire timestamp as the default output timestamp.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
