kennknowles commented on code in PR #27437:
URL: https://github.com/apache/beam/pull/27437#discussion_r1259853473
##########
model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/beam_runner_api.proto:
##########
@@ -327,6 +327,46 @@ message StandardPTransforms {
//
// Payload: none
TO_STRING = 8 [(beam_urn) = "beam:transform:to_string:v1"];
+
+ // Proto3 requires enum values to be in order, otherwise these would be
+ // at the top of the primitive transforms.
+
+ // DataSource is a Root Transform, and a source of data for downstream
+ // transforms in the same beam_fn_api.ProcessBundleDescriptor.
+ // It represents a logical PCollection.
+ //
+ // The DataSource transform is implemented in each SDK but not explicitly
+ // provided during pipeline construction. A runner inserts the transform
+ // in beam_fn_api.ProcessBundleDescriptors to indicate where the bundle
+ // can retrieve data for an associated beam_fn_api.ProcessBundleRequest.
+ // Data for the same request will be retrieved with the matching
instruction ID,
+ // and transform ID determined by the runner.
+ //
+ // The DataSource transform will take a stream of bytes from the remote
+ // source for the matching instruction ID and decode them as windowed
+ // values using the provided coder ID, which must be for a WINDOWED_VALUE
or
+ // PARAM_WINDOWED_VALUE coder.
+ //
+ // Payload: beam_fn_api.RemoteGrpcPort
+ DATA_SOURCE = 9 [(beam_urn) = "beam:runner:source:v1"];
Review Comment:
I think the reason this isn't in the runner API proto is that it is a fn api
detail, not a transform that has any meaning at the whole-graph level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
