Github user sethah commented on a diff in the pull request:
https://github.com/apache/spark/pull/17117#discussion_r104829816
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/clustering/KMeansSuite.scala ---
@@ -152,6 +158,35 @@ class KMeansSuite extends SparkFunSuite with
MLlibTestSparkContext with DefaultR
val kmeans = new KMeans()
testEstimatorAndModelReadWrite(kmeans, dataset,
KMeansSuite.allParamSettings, checkModelData)
}
+
+ test("training with initial model") {
+ val kmeans = new KMeans().setK(2).setSeed(1)
+ val model1 = kmeans.fit(rData)
+ val model2 =
kmeans.setInitMode("initialModel").setInitialModel(model1).fit(rData)
+ model2.clusterCenters.zip(model1.clusterCenters)
+ .foreach { case (center2, center1) => assert(center2 ~== center1
absTol 1E-8) }
+ }
+
+ test("training with initial model, error cases") {
+ val kmeans = new KMeans().setK(k).setSeed(1).setMaxIter(1)
+
+ // Sets initMode with 'initialModel', but does not specify initial
model.
+ intercept[IllegalArgumentException] {
--- End diff --
Yeah, I think the general idea laid out in the previous PR is preferable.
As you and DB say, if you'd like to make the second `.setInitMode` overwrite
the initial model then that is fine. With that change, then the behavior I
would prefer is:
````scala
test("params") {
val initialK = 3
val initialEstimator = new KMeans()
.setK(initialK)
val initialModel = initialEstimator.fit(dataset)
val km = new KMeans()
.setK(initialK + 1)
.setInitMode(MLlibKMeans.RANDOM)
assert(km.getK === initialK + 1)
assert(km.getInitMode === MLlibKMeans.RANDOM)
km.setInitialModel(initialModel)
// initialModel sets k and init mode
assert(km.getInitMode === MLlibKMeans.K_MEANS_INITIAL_MODEL)
assert(km.getK === initialK)
assert(km.getInitialModel.getK === initialK)
// setting k is ignored
km.setK(initialK + 1)
assert(km.getK === initialK)
// this should work since we already set initialModel
km.setInitMode(MLlibKMeans.K_MEANS_INITIAL_MODEL)
// changing initMode clears the initial model
km.setInitMode(MLlibKMeans.RANDOM)
assert(km.getInitMode === MLlibKMeans.RANDOM)
assert(!km.isSet(km.initialModel))
// k is retained from initial model
assert(km.getK === initialK)
// now k can be set
km.setK(initialK + 1)
assert(km.getK === initialK + 1)
// kmeans should throw an error since we shouldn't be allowed to set
init mode to "initialModel"
intercept[IllegalArgumentException] {
km.setInitMode(MLlibKMeans.K_MEANS_INITIAL_MODEL)
}
}
````
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
