HyukjinKwon commented on code in PR #38979:
URL: https://github.com/apache/spark/pull/38979#discussion_r1044247543
##########
connector/connect/common/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -305,6 +306,17 @@ message LocalRelation {
// Local collection data serialized into Arrow IPC streaming format which
contains
// the schema of the data.
bytes data = 1;
+
+ // (Optional) The user provided schema.
+ //
+ // The Sever side will update the column names and data types according to
this schema.
+ oneof schema {
+
+ DataType datatype = 2;
+
+ // Server will use Catalyst parser to parse this string to DataType.
+ string datatype_str = 3;
Review Comment:
Or we can always pass string implementation for now (by turning DataType to
a JSON representation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
