[
https://issues.apache.org/jira/browse/BEAM-10698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17186175#comment-17186175
]
Boyuan Zhang commented on BEAM-10698:
-------------------------------------
The root cause of this issue is we were using fake pubsub singal when translate
Impulse(), which gave use out-bounded element timestamp. Now we are translating
Impulse into inmemory source, where element timestamp is MIN_TIMESTAMP by
default.
Here is the translation logic:
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L766-L775
> SDFs broken for Dataflow runner v2 due to timestamps being out of bound
> -----------------------------------------------------------------------
>
> Key: BEAM-10698
> URL: https://issues.apache.org/jira/browse/BEAM-10698
> Project: Beam
> Issue Type: Bug
> Components: cross-language, io-py-kafka, runner-dataflow
> Reporter: Chamikara Madhusanka Jayalath
> Assignee: Boyuan Zhang
> Priority: P0
> Fix For: 2.24.0
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Seems like this is a regression introduced by
> [https://github.com/apache/beam/pull/11749]
> I have a short term fix here: [https://github.com/apache/beam/pull/12557]
> We should either.
> (1) Include [https://github.com/apache/beam/pull/12557] in the release branch
> (2) Find the underlying issue that produces wrong timestamps and do that fix
> (3) Revert [https://github.com/apache/beam/pull/11749] from the 2.24.0
> release branch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
