akashorabek commented on code in PR #34078:
URL: https://github.com/apache/beam/pull/34078#discussion_r1973160613
##########
it/google-cloud-platform/src/test/java/org/apache/beam/it/gcp/bigquery/BigQueryIOST.java:
##########
@@ -296,11 +297,10 @@ private void
generateDataAndWrite(BigQueryIO.Write<byte[]> writeIO) throws IOExc
.withSchema(schema)
.withCustomGcsTempLocation(ValueProvider.StaticValueProvider.of(tempLocation)));
- String runnerV2Experiment = "use_runner_v2";
String experiments =
configuration.writeMethod.equals(STORAGE_API_AT_LEAST_ONCE_METHOD)
- ? runnerV2Experiment + ",streaming_mode_at_least_once"
- : runnerV2Experiment;
+ ? GcpOptions.STREAMING_ENGINE_EXPERIMENT +
",streaming_mode_at_least_once"
+ : GcpOptions.STREAMING_ENGINE_EXPERIMENT;
Review Comment:
@damccorm The data duplication issue occurs even under small loads
specifically in the exactly_once method when using runner_v2. Can we avoid
using v2 for this method for now? I also noticed that in
[BigQueryIOLT](https://github.com/apache/beam/blob/25977b8b05c103b832d92593602699eb86e7ab04/it/google-cloud-platform/src/test/java/org/apache/beam/it/gcp/bigquery/BigQueryIOLT.java#L188),
we explicitly disabled v2, likely for the same reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
