chamikaramj commented on a change in pull request #14164:
URL: https://github.com/apache/beam/pull/14164#discussion_r593405175
##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/WriteFiles.java
##########
@@ -301,6 +321,21 @@
return toBuilder().setMaxNumWritersPerBundle(-1).build();
}
+ /**
+ * Returns a new {@link WriteFiles} that will write to the current {@link
FileBasedSink} with
+ * runner-determined sharding for unbounded data specifically. Currently
manual sharding is
+ * required for writing unbounded data with a fixed number of shards or a
predefined sharding
+ * function. This option allows the runners to get around that requirement
and perform automatic
+ * sharding.
+ *
+ * <p>Intended to only be used by runners. Users should use {@link
Review comment:
Ok, makes sense. I think a PTransformOverride should work for this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
