ueshin commented on code in PR #40238:
URL: https://github.com/apache/spark/pull/40238#discussion_r1123920584
##########
connector/connect/common/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -353,11 +353,16 @@ message LocalRelation {
optional bytes data = 1;
// (Optional) The schema of local data.
- // It should be either a DDL-formatted type string or a JSON string.
//
// The server side will update the column names and data types according to
this schema.
// If the 'data' is not provided, then this schema will be required.
- optional string schema = 2;
+ oneof schema {
Review Comment:
In some cases, we need to parse it beforehand anyway, for the case of
#40240, we can't create converter from the local python object to Arrow table
without the schema as `DataType` object.
Another example is UDFs. It needs to pickle the function, and its return
type as `DataType` object. We can't generate Python `DataType` object and
pickle it in the `command` field in server side anymore. So we need to parse
the DDL string beforehand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
