GitHub user nicklavers opened a pull request:
https://github.com/apache/spark/pull/14551
Fixed off-by-one error that biased randomizeInPlace
JIRA issue link:
https://issues.apache.org/jira/browse/SPARK-16961
Changed one line of Utils.randomizeInPlace to allow elements to stay in
place.
Created a unit test that runs a Pearson's chi squared test to determine
whether the output diverges significantly from a uniform distribution.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/VideoAmp/spark-public
SPARK-16961-randomizeInPlace
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14551.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14551
----
commit 4218c5e9a2ab727e54ad0c17a948712de321b145
Author: Nick Lavers <[email protected]>
Date: 2016-08-09T00:19:31Z
Fixed off-by-one error that biased randomizeInPlace against keeping
elements in place; added a seeded unit test for statistical significance
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
