amaliujia commented on code in PR #39084:
URL: https://github.com/apache/spark/pull/39084#discussion_r1051075228
##########
connector/connect/common/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -601,3 +602,19 @@ message Unpivot {
// (Required) Name of the value column.
string value_column_name = 5;
}
+
+message ToSchema {
+ // (Required) The input relation.
+ Relation input = 1;
+
+ // (Required) The user provided schema.
+ //
+ // The Sever side will update the dataframe with this schema.
+ oneof schema {
Review Comment:
This requires clients side to implement more. For example, client side
always convert StructType to string representation then Spark will convert it
back.
This basically asks clients to understand Spark's protocol on the string
based schema and then implement it right?
##########
connector/connect/common/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -601,3 +602,19 @@ message Unpivot {
// (Required) Name of the value column.
string value_column_name = 5;
}
+
+message ToSchema {
+ // (Required) The input relation.
+ Relation input = 1;
+
+ // (Required) The user provided schema.
+ //
+ // The Sever side will update the dataframe with this schema.
+ oneof schema {
Review Comment:
This requires clients side to implement more. For example, client side
always convert StructType to string representation then Spark will convert it
back.
This basically asks clients to understand Spark's protocol on the string
based schema and then implement it right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
