Guillaume Dardelet created SPARK-21268: ------------------------------------------
Summary: Redundant collectAsMap in KMeans Key: SPARK-21268 URL: https://issues.apache.org/jira/browse/SPARK-21268 Project: Spark Issue Type: Improvement Components: MLlib Affects Versions: 2.1.1, 2.1.0, 2.0.2, 2.0.1, 2.0.0 Reporter: Guillaume Dardelet Priority: Trivial As I was monitoring the perfomance of my algorithm with SparkUI, I noticed that their was a "collectAsMap" operation that was done hundreds of time at every iteration of Kmeans: https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/clustering/KMeans.scala#L295 It would work just as well by performing the following "foreach" on the RDD, and would slightly improve perfomance. Thoughts ? -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org