[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2017-01-09 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15730 Looks pretty good overall. Left one major comment about documentation and one about tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2017-01-02 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15730 @WeichenXu123 Thanks! Will take a look once I get back from vacation (in a week). Happy new year! --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2017-01-02 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15730 @brkyvz I update code and attach a running result screenshot, waiting for your review, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15730 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-12-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15730 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70574/ Test PASSed. ---

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15730 **[Test build #70574 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70574/testReport)** for PR 15730 at commit

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-12-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15730 **[Test build #70574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70574/testReport)** for PR 15730 at commit

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-12-11 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15730 @brkyvz All right, I'll update code ASAP. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-28 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15730 This way, all that needs to change in the implementation of multiply is: ```scala val intermediatePartitioner = new Partitioner { override def numPartitions: Int =

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-28 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15730 @WeichenXu123 How about if we only add: ```scala def multiply(other: BlockMatrix, numMidDimSplits: Integer): BlockMatrix ``` as the public API. Users shouldn't have to define

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-19 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15730 @brkyvz Good question about the shuffling data in the sparse case. Now I give some simple analysis on it (maybe not very strict): as discussed above, the shuffling data contains

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-18 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15730 Hi @WeichenXu123 Thank you for this PR. Sorry for taking so long to get back to you. Your optimization would be very helpful. I have a couple thoughts though. Your examples always take into account

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-02 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15730 ping @brkyvz who wrote these libraries. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15730 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15730 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67977/ Test PASSed. ---

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15730 **[Test build #67977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67977/consoleFull)** for PR 15730 at commit

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15730 **[Test build #67977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67977/consoleFull)** for PR 15730 at commit

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15730 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67969/ Test PASSed. ---

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15730 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15730 **[Test build #67969 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67969/consoleFull)** for PR 15730 at commit

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-02 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15730 @yanboliang @sethah I will be very pleasure to hear your opinions, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15730 **[Test build #67969 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67969/consoleFull)** for PR 15730 at commit