[
https://issues.apache.org/jira/browse/SPARK-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238673#comment-15238673
]
yuhao yang commented on SPARK-14154:
------------------------------------
result on 4 cores:
||scale ||old ||new ||ratio ||
|1M |1.01 |1.29 |1.277227723 |
|10M |2.89 |3.73 |1.290657439 |
|50M |12.34 |16.06 |1.301458671 |
|100M |40 |52 |1.3 |
[~mengxr] The speed did get slower by about 30%. Sorry about it, I can prepare
a PR to revert the change.
> Simplify the implementation for Kolmogorov–Smirnov test
> -------------------------------------------------------
>
> Key: SPARK-14154
> URL: https://issues.apache.org/jira/browse/SPARK-14154
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Reporter: yuhao yang
> Assignee: yuhao yang
> Priority: Critical
> Fix For: 2.0.0
>
>
> I just read the code for KolmogorovSmirnovTest and find it could be much
> simplified following the original definition.
> Send a PR for discussion
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]