Github user ConeyLiu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19661#discussion_r150172493
--- Diff:
core/src/test/scala/org/apache/spark/serializer/KryoSerializerSuite.scala ---
@@ -108,6 +108,27 @@ class KryoSerializerSuite extends SparkFunSuite with
SharedSparkContext {
check(Array(Array("1", "2"), Array("1", "2", "3", "4")))
}
+ test("safely register class for mllib/ml") {
+ val conf = new SparkConf(false)
+ val ser = new KryoSerializer(conf)
+
+ Seq("org.apache.spark.mllib.linalg.Vector",
+ "org.apache.spark.mllib.linalg.DenseVector",
+ "org.apache.spark.mllib.linalg.SparseVector",
+ "org.apache.spark.mllib.linalg.Matrix",
+ "org.apache.spark.mllib.linalg.DenseMatrix",
+ "org.apache.spark.mllib.linalg.SparseMatrix",
+ "org.apache.spark.ml.linalg.Vector",
+ "org.apache.spark.ml.linalg.DenseVector",
+ "org.apache.spark.ml.linalg.SparseVector",
+ "org.apache.spark.ml.linalg.Matrix",
+ "org.apache.spark.ml.linalg.DenseMatrix",
+ "org.apache.spark.ml.linalg.SparseMatrix",
+ "org.apache.spark.ml.feature.Instance",
+ "org.apache.spark.ml.feature.OffsetInstance"
+ ).foreach(!Utils.classIsLoadable(_))
--- End diff --
This just want to indicate we didn't introduce extra jar dependency. I can
delete it if it's unnecessary.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
