scwhittle commented on code in PR #37683:
URL: https://github.com/apache/beam/pull/37683#discussion_r2840685838
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java:
##########
@@ -1309,6 +1320,8 @@ public DataflowPipelineJob run(Pipeline pipeline) {
if (!experiments.contains("enable_windmill_service")) {
experiments.add("enable_windmill_service");
}
+ // TODO: The experiments added above are not included in the list of
experiments sent to the
Review Comment:
should we just remove the experiments settings here? I think that the
options.setEnabledStreamingEngine probably is sufficient.
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java:
##########
@@ -1299,6 +1299,17 @@ public DataflowPipelineJob run(Pipeline pipeline) {
if (shouldActAsStreaming(pipeline)) {
options.setStreaming(true);
+ {
+ List<String> experiments =
+ options.getExperiments() == null
+ ? new ArrayList<>()
+ : new ArrayList<>(options.getExperiments());
+ // Experiment marking that the harness supports tag encoding v2
+ // Backend will enable tag encoding v2 only if the harness supports it.
+ experiments.add("streaming_engine_state_tag_encoding_v2_supported");
Review Comment:
should we guard doing this based upon the compatabilityversion option?
IT could help in case someone starts on 2.Y with compatabilityversion 2.X
because they think they might want to rollback to 2.X
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
