chamikaramj commented on code in PR #37376:
URL: https://github.com/apache/beam/pull/37376#discussion_r2729459282
##########
model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/schema.proto:
##########
@@ -237,3 +237,23 @@ message MapTypeEntry {
message LogicalTypeValue {
FieldValue value = 1;
}
+
+// Information needed to represent a Coder of type SCHEMA.
+message SchemaCoderPayload {
+ // The schema to use for encoding corresponding Row types.
+ Schema schema = 1;
+
+ // Function mapping from underlying object to Row type.
+ FunctionSpec to_row_fn = 2;
+
+ // Function mapping from Row type to underlying object.
+ FunctionSpec from_row_fn = 3;
Review Comment:
Is the expectation that this function spec is only understandable by a given
SDK. If so I have a couple of questions.
(1) How can one (a runner for example) identify the correct SDK given the
coder ? Should we also attach the environment ID to the coder ? (I don't think
just URL in the function spec is adequate).
(2) Does this mean that this coder is not truly portable ? For example,
cannot be used in a cross-language boundary ? To make it truly portable, we
might need a list of FunctionSpecs with entries for multiple environments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
