[
https://issues.apache.org/jira/browse/BEAM-10698?focusedWorklogId=470541&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-470541
]
ASF GitHub Bot logged work on BEAM-10698:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Aug/20 02:43
Start Date: 14/Aug/20 02:43
Worklog Time Spent: 10m
Work Description: boyuanzz commented on a change in pull request #12577:
URL: https://github.com/apache/beam/pull/12577#discussion_r470379176
##########
File path:
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java
##########
@@ -1047,7 +1047,9 @@ public void setMaxReadTime(Long maxReadTime) {
// "beam_fn_api_use_deprecated_read" is not enabled.
if (!ExperimentalOptions.hasExperiment(input.getPipeline().getOptions(),
"beam_fn_api")
|| ExperimentalOptions.hasExperiment(
- input.getPipeline().getOptions(),
"beam_fn_api_use_deprecated_read")) {
+ input.getPipeline().getOptions(),
"beam_fn_api_use_deprecated_read")
+ || !ExperimentalOptions.hasExperiment(
+ input.getPipeline().getOptions(), "use_sdf_kafka_read")) {
Review comment:
I couldn't find a way to pass experiments into ExpansionService:
https://github.com/apache/beam/blob/master/sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/ExpansionService.java#L384-L393
so I just leave the test to verity non-sdf read.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 470541)
Time Spent: 1h 50m (was: 1h 40m)
> SDFs broken for Dataflow runner v2 due to timestamps being out of bound
> -----------------------------------------------------------------------
>
> Key: BEAM-10698
> URL: https://issues.apache.org/jira/browse/BEAM-10698
> Project: Beam
> Issue Type: Bug
> Components: cross-language, io-py-kafka, runner-dataflow
> Reporter: Chamikara Madhusanka Jayalath
> Assignee: Boyuan Zhang
> Priority: P0
> Fix For: 2.24.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Seems like this is a regression introduced by
> [https://github.com/apache/beam/pull/11749]
> I have a short term fix here: [https://github.com/apache/beam/pull/12557]
> We should either.
> (1) Include [https://github.com/apache/beam/pull/12557] in the release branch
> (2) Find the underlying issue that produces wrong timestamps and do that fix
> (3) Revert [https://github.com/apache/beam/pull/11749] from the 2.24.0
> release branch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
