zhengruifeng commented on code in PR #49569:
URL: https://github.com/apache/spark/pull/49569#discussion_r1921980200
##########
mllib/src/main/resources/META-INF/services/org.apache.spark.ml.Transformer:
##########
@@ -18,3 +18,16 @@
# Spark Connect ML uses ServiceLoader to find out the supported Spark Ml
non-model transformer.
# So register the supported transformer here if you're trying to add a new one.
org.apache.spark.ml.feature.VectorAssembler
+
+########### Model for loading
+# classification
+org.apache.spark.ml.classification.LogisticRegressionModel
+org.apache.spark.ml.classification.DecisionTreeClassificationModel
+org.apache.spark.ml.classification.RandomForestClassificationModel
+org.apache.spark.ml.classification.GBTClassificationModel
+# regression
+
Review Comment:
```suggestion
# regression
```
##########
sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ml/MLUtils.scala:
##########
@@ -364,16 +364,22 @@ private[ml] object MLUtils {
def load(sessionHolder: SessionHolder, className: String, path: String):
Object = {
Review Comment:
btw, it seems a non-model transformer is not allowed to load in
`proto.MlCommand.CommandCase.READ`?
##########
sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ml/MLUtils.scala:
##########
@@ -364,16 +364,22 @@ private[ml] object MLUtils {
def load(sessionHolder: SessionHolder, className: String, path: String):
Object = {
Review Comment:
I see this `load` is only used to handle `proto.MlCommand.CommandCase.READ`,
and the type is already given `MODEL` `ESTIMATOR` and `EVALUATOR`.
What about split it to 3 functions? e.g. `loadEstimator`
##########
mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala:
##########
@@ -105,6 +105,11 @@ private[ml] object Node {
split = Split.fromOld(oldNode.split.get, categoricalFeatures),
impurityStats = null)
}
}
+
+ // Create a dummy node used for ml connect only
+ def dummyNode: Node = {
Review Comment:
```suggestion
val dummyNode = {
```
nit, avoid recreating it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
