Julian Jorczik created SPARK-34664: -------------------------------------- Summary: Provide silhouette score for each sample when using ClusteringEvaluator Key: SPARK-34664 URL: https://issues.apache.org/jira/browse/SPARK-34664 Project: Spark Issue Type: Improvement Components: ML Affects Versions: 3.1.1 Reporter: Julian Jorczik
Computing the average silhouette score is already implemented when using ClusteringEvaluator. When looking at the [source code|https://gitlab.com/mark91/SparkClusteringEvaluationMetrics/-/blob/master/src/main/scala/org/apache/spark/ml/evaluation/SquaredEuclideanSilhouetteEvaluator.scala] of ClusteringEvaluator, I think it would be easy to provide not only the average silhouette score but also the silhouette score for each sample, as they are already computed (Line 95-99). The silhouette score for each sample can be helpful to generate a silhouette plot for instance as described in [this scikit-learn article|https://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_silhouette_analysis.html]. The resulting feature would be equivalent to the silhouette_samples function implemented in scikit-learn. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org