[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-30 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-98002797 @debasish83 Do you mind me sending you an update? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-30 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-98021307 @mengxr please go ahead... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-26 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-96403986 was very last few weeks...update it in next few days... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-21 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-95009893 @debasish83 Do you have time to update the PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-07 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-90740329 @debasish83 In RankingMetrics, the label set doesn't have an ordering. It is either hit or miss at each position. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-05 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-89728639 @debasish83 do you mean RMSE? it is well-defined but not very useful. MAP is the useful metric. I think that only a rank-dependent metric makes sense. --- If your

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-05 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-89729377 I meant MAP...what's the MAP on netflix dataset you have seen before and with what lambda ? I am running MAP experiments with various factorization formulations

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-05 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-89729632 I don't remember from the last time I tried the Netflix dataset but in general MAP is low, like 0.05-0.1, even when recommendations are good according to RMSE.

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-05 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-89729777 agreed with the implicit MAP calculationFor netflix dataset, I got 0.014...May be I need to use a better regularization...was that 0.05-0.1 number from using

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-05 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-89729865 That might be a fine score. I remember them being MAP being quite low in general and that 0.1 would be very good. I don't recall the number for Netflix, so don't take

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-04 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27769592 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -167,23 +169,66 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-04 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-89697236 @srowen For netflix dataset what's the MAP you have seen before...I started experiments on Netflix dataset...lambda is 0.065 for netflix as well right ? For MovieLens

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-04 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-89706247 @coderxiang @mengxr If I have a dataset with implicit (click or 0) then MAP is not that well defined right since in label set everything is 1.0 and so there is no

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707377 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -138,14 +141,122 @@ class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707375 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -138,14 +141,122 @@ class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707380 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -138,14 +141,122 @@ class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707373 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -138,14 +141,122 @@ class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707386 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -138,14 +141,122 @@ class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707370 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -138,14 +141,122 @@ class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707366 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -70,26 +71,28 @@ class MatrixFactorizationModel(

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707360 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -167,23 +169,66 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707363 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -167,23 +169,66 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707367 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -70,26 +71,28 @@ class MatrixFactorizationModel(

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707364 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -167,23 +169,66 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707362 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -167,23 +169,66 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27707356 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -167,23 +169,66 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-89083630 @debasish83 The code style still need fixes. On the high level, I think having variable `num` is not necessary. Let's do a global num in this PR and see whether users ask

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-02 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27712646 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -138,14 +141,122 @@ class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27624736 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class MatrixFactorizationModel

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-04-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27624801 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -39,14 +39,15 @@ import org.apache.spark.rdd.RDD object

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88312094 [Test build #29518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29518/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88312097 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88311999 [Test build #29518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29518/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88346990 I reran the map computation on MovieLens with varying ranks: Example run: ./bin/spark-submit --master spark://TUSCA09LMLVT00C.local:7077 --class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88346961 [Test build #29521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29521/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88311843 [Test build #29517 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29517/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88311846 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88311599 [Test build #29517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29517/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88347022 @mengxr could you please do another passI might have missed the JavaRDD compatibility issue but fixed rest of your comments... --- If your project is set up for

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88361341 [Test build #29521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29521/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88361354 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27533769 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27525485 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -74,6 +75,9 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27528071 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27528120 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27528991 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27528959 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27525568 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27528198 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27528238 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27529347 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -35,33 +41,33 @@ import

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27529308 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -35,33 +41,33 @@ import

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27529218 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -17,14 +17,20 @@ package

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27529231 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -17,14 +17,20 @@ package

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27529681 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r27535273 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88291470 @mengxr I also added 2 test-cases for batch predict APIs. These features are useful if users are interested in computing MAP measures...Let me know if I move the

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-31 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-88292172 If we move computeRankingMetrics and computeRMSE to a better place, I can guard it through tests... --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-03-16 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r26545110 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -18,14 +18,14 @@ package org.apache.spark.examples.mllib

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955630 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -74,6 +75,9 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955624 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -18,14 +18,14 @@ package org.apache.spark.examples.mllib

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955687 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class MatrixFactorizationModel

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955679 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class MatrixFactorizationModel

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955664 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -17,14 +17,20 @@ package

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955654 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955648 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955659 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955670 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -17,14 +17,20 @@ package

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955680 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class MatrixFactorizationModel

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955662 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955657 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955652 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955678 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class MatrixFactorizationModel

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955650 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -171,18 +175,62 @@ object MovieLensALS {

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955690 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class MatrixFactorizationModel

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955684 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class MatrixFactorizationModel

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955626 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -39,14 +39,15 @@ import org.apache.spark.rdd.RDD object

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955674 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -35,33 +41,33 @@ import org.apache.spark.rdd.RDD

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955675 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -103,13 +109,106 @@ class MatrixFactorizationModel

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2015-02-18 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r24955671 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -35,33 +41,33 @@ import org.apache.spark.rdd.RDD

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-27 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-64804819 @mengxr can we please review and merge this ? It will greatly help in reporting results from PLSA and other variants of matrix factorization results...should I update

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-21 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-63979463 More details about the API added and experiments are on the JIRA --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-63722805 [Test build #23632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23632/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-19 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r20616949 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -39,14 +39,15 @@ import org.apache.spark.rdd.RDD object

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-19 Thread debasish83
Github user debasish83 commented on a diff in the pull request: https://github.com/apache/spark/pull/3098#discussion_r20617279 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala --- @@ -18,14 +18,14 @@ package org.apache.spark.examples.mllib

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-63736377 [Test build #23642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23642/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-63738984 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-63738974 **[Test build #23632 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23632/consoleFull)** for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-63745215 [Test build #23642 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23642/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-63745222 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-62670490 [Test build #23246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23246/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-11 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-62670769 @mengxr added recommendAll API to MatrixFactorizationModel and right now the catesian based topK finding is also in the code for validation... Example run:

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-11 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-62670877 More performance tests undergoing for internal datasets where the cartesian code is really slow (due to groupByKey)...For MovieLens there is no substantial

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-62675915 [Test build #23246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23246/consoleFull) for PR 3098 at commit

[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

2014-11-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3098#issuecomment-62675921 Test PASSed. Refer to this link for build results (access rights to CI server needed):