[
https://issues.apache.org/jira/browse/SPARK-8587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600237#comment-14600237
]
Joseph K. Bradley commented on SPARK-8587:
------------------------------------------
I agree; we should not change the behavior of the existing function, and we
will need to maintain matching APIs for Scala/Java and Python. I think this
will be easily supported within the Pipelines API, where KMeans is currently
being added: [SPARK-7879]. The initial PR will add only a prediction column
(predicted cluster), but a follow-up could add a column of costs or of soft/raw
predictions (which could be 1/cost).
Would you be able to help out with this extension of Pipelines, once the
initial PR gets in? If so, we could close this JIRA and PR for now. Thanks!
> Return cost and cluster index KMeansModel.predict
> -------------------------------------------------
>
> Key: SPARK-8587
> URL: https://issues.apache.org/jira/browse/SPARK-8587
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Reporter: Sam Stoelinga
> Priority: Minor
>
> Looking at PySpark the implementation of KMeansModel.predict
> https://github.com/apache/spark/blob/master/python/pyspark/mllib/clustering.py#L102
> :
> Currently:
> it calculates the cost of the closest cluster and returns the index only.
> My expectation:
> Easy way to let the same function or a new function to return the cost with
> the index.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
