Github user ZakariaHili commented on a diff in the pull request:
https://github.com/apache/spark/pull/15965#discussion_r92402189
--- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala
---
@@ -306,12 +307,20 @@ class KMeans @Since("1.5.0") (
@Since("2.0.0")
override def fit(dataset: Dataset[_]): KMeansModel = {
+ val handlePersistence = dataset.rdd.getStorageLevel ==
StorageLevel.NONE
+ fit(dataset, handlePersistence)
+ }
+
+ @Since("2.2.0")
+ protected def fit(dataset: Dataset[_], handlePersistence: Boolean):
KMeansModel = {
--- End diff --
Hi @jkbradley,
In actually, I thought of the same thing, but Mr. yuhao yang advised me to
do the same style of LogisticRegression.scala
anywhere I changed the file as you want, and I changed the code a little
(e.g : no need to generate the rdd two times
the first one to check storage level and the second one for Kmeans
)
here the change
https://github.com/ZakariaHili/spark/commit/58de5499d0930e4019c94718a2a9ed3dd715acb2
do you still want from me to send a PR ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
