[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-19 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14551 Merged to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64025/ Test PASSed. ---

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #64025 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64025/consoleFull)** for PR 14551 at commit [`17d28a5`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #64025 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64025/consoleFull)** for PR 14551 at commit [`17d28a5`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64012/ Test FAILed. ---

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #64012 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64012/consoleFull)** for PR 14551 at commit [`76efd0b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #64012 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64012/consoleFull)** for PR 14551 at commit [`76efd0b`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63999/ Test FAILed. ---

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #63999 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63999/consoleFull)** for PR 14551 at commit [`4848f42`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #63999 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63999/consoleFull)** for PR 14551 at commit [`4848f42`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14551 @nicklavers I agree it's not great to assume that the current output is correct, though I strongly suspect it is. We'd ideally do some analysis to understand what the expected range of outcomes are

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63946/ Test FAILed. ---

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #63946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63946/consoleFull)** for PR 14551 at commit [`42e750f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #63946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63946/consoleFull)** for PR 14551 at commit [`42e750f`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-17 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 Okay I'll push a new commit --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-17 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/14551 @nicklavers it is a key test. The behavior change should be because you changed the random function, which causes the output changing. I agree with @srowen you just have to adjust your expected

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-17 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 So if `model.gaussiansDF.show()` is testing a key behavior of the model, then I don't know what it is, and I don't feel comfortable just changing the test to make it pass, since that seems to def

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-17 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14551 I'd expect the Gaussians to move as a result of this change. I think it's fair to simply modify the test to match the current output in this case. --- If your project is set up for it, you can repl

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-16 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/14551 `model.gaussiansDF.show()` displays the `mean` and `variance` of the gaussians, which are Dataframes. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-16 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 and `KMeans` is passing for me --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-16 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 Okay so I've tried a handful of seeds, and none have produced the results that the test is expecting. The doctests that are failing are `model.gaussiansDF.show()`, which as far as I can tell is j

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-15 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14551 If we can get that last test failure patched in any reasonable form, I'll merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14551 Jenkins add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #63630 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63630/consoleFull)** for PR 14551 at commit [`4bb0afe`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-11 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14551 @nicklavers Please also change seed for ```GaussianMixture``` doctest in ```python/pyspark/ml/clustering.py```. And check whether we need to change seed for ```KMeans``` doctest. Thanks. --- If

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63630/ Test FAILed. ---

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #63630 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63630/consoleFull)** for PR 14551 at commit [`4bb0afe`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-11 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-11 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 hey @srowen can you start up a new test? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-11 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 These seeds passed the tests on my machine, at least --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-11 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 Okay I'll find some new seeds and commit again --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-11 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14551 +1 @srowen I think we can change to another random seed to make it pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14551 Anyway, back to the actual content -- the actual test assertions in the failing test are correct, in that it's obvious that the first two points and second two points should cluster together. I expec

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 The doctests in `python/pyspark/mllib/clustering.py` will have to be changed, too --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 I mean, I can certainly reverse-engineer a test that passes, but it wouldn't be very valuable, since I don't know enough about how `GaussianMixture` is supposed to work to write a good test for i

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14551 You're probably right, and I think it's OK to change the test to match the current output as a result. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 Hey @yanboliang, I'm failing `test_gmm` in `python/pyspark/mllib/tests.py`, and I'm fairly certain it's because the test relies on the bug I'm fixing. I saw you worked on `GaussianMixture

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 Okay I'm looking into this pyspark test failure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63521/ Test FAILed. ---

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14551 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #63521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63521/consoleFull)** for PR 14551 at commit [`e33741c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14551 **[Test build #63521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63521/consoleFull)** for PR 14551 at commit [`e33741c`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14551 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-10 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14551 Anyway the test and change look sound, let's test it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark issue #14551: [SPARK-16961][CORE] Fixed off-by-one error that biased r...

2016-08-08 Thread nicklavers
Github user nicklavers commented on the issue: https://github.com/apache/spark/pull/14551 Thanks @HyukjinKwon! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes