hvanhovell commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1606482118
##########
connector/connect/common/src/main/protobuf/spark/connect/commands.proto:
##########
@@ -484,3 +486,26 @@ message CreateResourceProfileCommandResult {
// (Required) Server-side generated resource profile id.
int32 profile_id = 1;
}
+
+// Command to remove `CashedRemoteRelation`
+message RemoveCachedRemoteRelationCommand {
+ // (Required) The remote to be related
+ CachedRemoteRelation relation = 1;
+}
+
+message CheckpointCommand {
+ // (Required) The logical plan to checkpoint.
+ Relation relation = 1;
+
+ // (Optional) Locally checkpoint using a local temporary
+ // directory in Spark Connect server (Spark Driver)
+ optional bool local = 2;
+
+ // (Optional) Whether to checkpoint this dataframe immediately.
+ optional bool eager = 3;
Review Comment:
The default is `eager = true` right? Should the protocol encode this better?
Currently the protocol defaults to `eager = false` if the field is not set, so
my question is should we flip the logic (i.e. replace this with `lazy`) so the
default behavior does not require you to set additional fields.
The same question for `local`...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
