sryza commented on code in PR #54104:
URL: https://github.com/apache/spark/pull/54104#discussion_r2760543361
##########
sql/connect/common/src/main/protobuf/spark/connect/pipelines.proto:
##########
@@ -224,6 +253,7 @@ message PipelineCommand {
}
}
+
Review Comment:
nit: remove
##########
sql/connect/common/src/main/protobuf/spark/connect/pipelines.proto:
##########
@@ -165,6 +167,33 @@ message PipelineCommand {
}
}
+ // Request to execute all flows for a single table remotely.
+ message ExecuteOutput {
+
+ // The output (table or materialized view) definition.
+ optional DefineOutput define_output = 1;
+
+ // The flows to execute for this table.
+ repeated DefineFlow define_flows = 2;
+
+ // Whether to perform a full refresh instead of an incremental update.
+ optional bool full_refresh = 3;
+
+ // The catalog to use for execution.
+ optional string catalog = 4;
+
+ // The database to use for execution.
+ optional string database = 5;
+
+ // SQL configurations for this execution.
+ map<string, string> sql_conf = 7;
+
+ // Storage location for pipeline checkpoints and metadata.
+ optional string storage = 8;
+
+ google.protobuf.Any extension = 999;
Review Comment:
It's conventional to make this repeated in case there are multiple
extensions.
##########
sql/connect/common/src/main/protobuf/spark/connect/pipelines.proto:
##########
@@ -165,6 +167,33 @@ message PipelineCommand {
}
}
+ // Request to execute all flows for a single table remotely.
Review Comment:
```suggestion
// Request to execute all flows for a single output (dataset or sink)
remotely.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
