akashorabek commented on code in PR #34078:
URL: https://github.com/apache/beam/pull/34078#discussion_r1973160613
##########
it/google-cloud-platform/src/test/java/org/apache/beam/it/gcp/bigquery/BigQueryIOST.java:
##########
@@ -296,11 +297,10 @@ private void
generateDataAndWrite(BigQueryIO.Write<byte[]> writeIO) throws IOExc
.withSchema(schema)
.withCustomGcsTempLocation(ValueProvider.StaticValueProvider.of(tempLocation)));
- String runnerV2Experiment = "use_runner_v2";
String experiments =
configuration.writeMethod.equals(STORAGE_API_AT_LEAST_ONCE_METHOD)
- ? runnerV2Experiment + ",streaming_mode_at_least_once"
- : runnerV2Experiment;
+ ? GcpOptions.STREAMING_ENGINE_EXPERIMENT +
",streaming_mode_at_least_once"
+ : GcpOptions.STREAMING_ENGINE_EXPERIMENT;
Review Comment:
@damccorm The data duplication issue occurs even under small loads
specifically in the `exactly_once` method when using `runner_v2`. Can we avoid
using `v2` for this method for now? I also noticed that in
[BigQueryIOLT](https://github.com/apache/beam/blob/25977b8b05c103b832d92593602699eb86e7ab04/it/google-cloud-platform/src/test/java/org/apache/beam/it/gcp/bigquery/BigQueryIOLT.java#L188),
we explicitly disabled `v2`, likely for the same reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
