[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-15 Thread ankurdave
Github user ankurdave commented on the issue: https://github.com/apache/spark/pull/16271 Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-15 Thread ankurdave
Github user ankurdave commented on the issue: https://github.com/apache/spark/pull/16271 Thanks @aray for the explanation. I agree with @srowen - this looks reasonable to me. I'm going to merge it. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-15 Thread aray
Github user aray commented on the issue: https://github.com/apache/spark/pull/16271 Yes the improvement is from the sum of magnitudes of initial values being closer to the (known) sum of the solution. Fiddling with resetProb controls a completely different thing. The current

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-14 Thread aray
Github user aray commented on the issue: https://github.com/apache/spark/pull/16271 **References** [Pagerank paper](http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf) > We need to make an initial assignment of the ranks. This assignment can be made by one of several

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-14 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16271 I just emailed @ankurdave and he is going to look at this tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-14 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16271 I am not sure who if anyone would review graphx at this point, and I am not so familiar with the implementation here. If it converges to the same answer faster that's good. it might be nice to

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16271 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70139/ Test PASSed. ---

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16271 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16271 **[Test build #70139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70139/consoleFull)** for PR 16271 at commit

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16271 **[Test build #70139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70139/consoleFull)** for PR 16271 at commit

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-14 Thread aray
Github user aray commented on the issue: https://github.com/apache/spark/pull/16271 ping @srowen @dbtsai @rxin @ankurdave @jegonzal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-14 Thread aray
Github user aray commented on the issue: https://github.com/apache/spark/pull/16271 Updated the above benchmark code with a log normal random graph on 10,000 vertices the difference is much more drastic. ![](http://i.imgur.com/Zo56dEO.png) (take the very bottom of the graph

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16271 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70101/ Test PASSed. ---

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16271 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16271 **[Test build #70101 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70101/consoleFull)** for PR 16271 at commit

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16271 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16271 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70100/ Test PASSed. ---

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16271 **[Test build #70100 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70100/consoleFull)** for PR 16271 at commit

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16271 **[Test build #70101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70101/consoleFull)** for PR 16271 at commit

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16271 **[Test build #70100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70100/consoleFull)** for PR 16271 at commit