GitHub user dorx opened a pull request:
https://github.com/apache/spark/pull/1710
[SPARK-2782][mllib] Bug fix for getRanks in SpearmanCorrelation
getRanks computes the wrong rank when numPartition >= size in the input
RDDs before this patch. added units to address this bug.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dorx/spark correlationBug
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1710
----
commit 043ff8327bd3520319e3615edc83b4bb435b301d
Author: Doris Xin <[email protected]>
Date: 2014-08-01T03:03:53Z
bug fix for spearman corner case
where numPartition >= size in the input RDDs
commit 31db920b667e30d3043469f183b03aabcdaf25d6
Author: Doris Xin <[email protected]>
Date: 2014-08-01T03:11:00Z
revert unnecessary change
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
