damccorm commented on code in PR #40713:
URL: https://github.com/apache/beam/pull/40713#discussion_r4352648283
##########
sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtil.java:
##########
@@ -127,16 +127,41 @@ public static boolean isWildcard(GcsPath spec) {
return GcsPath.isWildcard(spec);
}
+ /**
+ * Experiment that falls back to GcsUtil V1 (gcsio). V2 (java-storage) is
the default, and this
+ * experiment is the escape hatch should V2 misbehave for a pipeline.
+ */
+ public static final String DISABLE_GCSUTIL_V2_EXPERIMENT =
"disable_gcsutil_v2";
+
+ /**
+ * Experiment that used to opt into GcsUtil V2. V2 is now the default, so
this is a no-op kept so
+ * that existing pipelines that still pass it keep working. {@link
#DISABLE_GCSUTIL_V2_EXPERIMENT}
+ * takes precedence if both are set.
+ */
+ public static final String USE_GCSUTIL_V2_EXPERIMENT = "use_gcsutil_v2";
+
+ /**
+ * Experiment that makes GcsUtil V1 talk to GCS over gRPC. Only V1 (gcsio)
supports it, so it also
+ * falls back to V1, the same as {@link #DISABLE_GCSUTIL_V2_EXPERIMENT}.
+ */
+ public static final String USE_GRPC_FOR_GCS_EXPERIMENT = "use_grpc_for_gcs";
+
GcsUtil(PipelineOptions options) {
this.delegate = new GcsUtilV1.GcsUtilFactory().create(options);
- if (ExperimentalOptions.hasExperiment(options, "use_gcsutil_v2")) {
+ @Nullable String v1Experiment = null;
+ if (ExperimentalOptions.hasExperiment(options,
DISABLE_GCSUTIL_V2_EXPERIMENT)) {
+ v1Experiment = DISABLE_GCSUTIL_V2_EXPERIMENT;
+ } else if (ExperimentalOptions.hasExperiment(options,
USE_GRPC_FOR_GCS_EXPERIMENT)) {
+ v1Experiment = USE_GRPC_FOR_GCS_EXPERIMENT;
Review Comment:
Do we plan to enable grpc with v2 ever? Or why is this scoped out?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
