WeichenXu123 commented on code in PR #40297:
URL: https://github.com/apache/spark/pull/40297#discussion_r1127830764
##########
connector/connect/common/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -81,13 +82,50 @@ message Relation {
// Catalog API (experimental / unstable)
Catalog catalog = 200;
+ // ML relation
+ MlRelation ml_relation = 300;
+
// This field is used to mark extensions to the protocol. When plugins
generate arbitrary
// relations they can add them here. During the planning the correct
resolution is done.
google.protobuf.Any extension = 998;
Unknown unknown = 999;
}
}
+message MlRelation {
+ oneof ml_relation_type {
+ ModelTransform model_transform = 1;
+ FeatureTransform feature_transform = 2;
+ ModelAttr model_attr = 3;
+ ModelSummaryAttr model_summary_attr = 4;
+ }
+ message ModelTransform {
+ Relation input = 1;
+ int64 model_ref_id = 2;
+ Params params = 3;
+ }
+ message FeatureTransform {
Review Comment:
Got it. Then we can treat it as normal transformer because it only contains
splits param
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
