[
https://issues.apache.org/jira/browse/BEAM-12931?focusedWorklogId=654284&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-654284
]
ASF GitHub Bot logged work on BEAM-12931:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Sep/21 22:40
Start Date: 22/Sep/21 22:40
Worklog Time Spent: 10m
Work Description: laraschmidt commented on a change in pull request
#15540:
URL: https://github.com/apache/beam/pull/15540#discussion_r714358223
##########
File path:
runners/core-java/src/main/java/org/apache/beam/runners/core/SimpleDoFnRunner.java
##########
@@ -1190,12 +1190,12 @@ public Timer withOutputTimestamp(Instant
outputTimestamp) {
* </ul>
*/
private void setAndVerifyOutputTimestamp() {
-
if (outputTimestamp != null) {
checkArgument(
- !outputTimestamp.isBefore(elementInputTimestamp),
Review comment:
So I ran the new validates runner tests against FnApiDoFnRunner. Turns
out it was not properly accounting for DoFN skew when setting timers. But it
wasn't doing any checking on timestamps at all for normal DoFn emit.
I added checks for the normal DoFn case and allowed for skew in the timer
case. It was a bit more extensive change then I expected so PTAL. If these
checks make sense I can add some unit testing for the FnApiRunner, though most
of the tests seem focused on splitting and not other functionality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 654284)
Time Spent: 0.5h (was: 20m)
> Timer.setOutputTimestamp doesn't take into account for
> DoFn#getAllowedTimestampSkew()
> -------------------------------------------------------------------------------------
>
> Key: BEAM-12931
> URL: https://issues.apache.org/jira/browse/BEAM-12931
> Project: Beam
> Issue Type: Bug
> Components: beam-model
> Reporter: Lara Schmidt
> Assignee: Lara Schmidt
> Priority: P2
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> A DoFn may emit elements with a timestamp up to
> DoFn#getAllowedTimestampSkew() before the current element's timestamp.
> However getAllowedTimestampSkew is not properly accounted for in looking at
> the output timestamp of a timer.
> Context:
> [https://lists.apache.org/thread.html/r7554658114ddde86c5d82e1c39fe7e1ef587fe926b8e406d1130d501%40%3Cdev.beam.apache.org%3E]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
