amaliujia commented on code in PR #38086:
URL: https://github.com/apache/spark/pull/38086#discussion_r998920195
##########
connector/connect/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -65,11 +65,21 @@ message SQL {
message Read {
oneof read_type {
NamedTable named_table = 1;
+ DataSource data_source = 2;
}
message NamedTable {
repeated string parts = 1;
}
+
+ message DataSource {
+ // Required. Supported formats include: parquet, orc, text, json, parquet,
csv, avro.
Review Comment:
I really like how Apache Beam document their proto and I want to match it in
connect once the proto becomes stable.
So this part will be expanded anyway (e.g. include the full list, document
case sensitivity, document applicable options for each format if there is any,
etc.)
##########
connector/connect/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -65,11 +65,21 @@ message SQL {
message Read {
oneof read_type {
NamedTable named_table = 1;
+ DataSource data_source = 2;
}
message NamedTable {
repeated string parts = 1;
}
+
+ message DataSource {
+ // Required. Supported formats include: parquet, orc, text, json, parquet,
csv, avro.
Review Comment:
I really like how Apache Beam document their proto and I want to match it in
connect once the proto becomes stable:
https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/beam_runner_api.proto
So this part will be expanded anyway (e.g. include the full list, document
case sensitivity, document applicable options for each format if there is any,
etc.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
