Github user coderh commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-45898782
Ok, I have found the error in my metric.
```
val itemFactors = model.productFeatures.collect()
```
This line is for creating a item-factor matrix, the
Github user coderh commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-45731942
I have tried different lamdba and # features. But nothing has changed. To
be clear, initially, the Movielens dataset it is divided into training set(80%)
and test
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-45792263
You mentioned trying lots of values but what did you try? What about other
test metrics -- to rule out some problem in the evaluation? Maybe you can share
some of how you
Github user coderh commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-45640620
I have recently tested expected percentile rank evaluation method proposed
in the paper on the Movielens data set and a real world data set. However, I
got a expected rank
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-45647190
The results depend a whole lot on the choice of parameters. Did you try
some degree of search for the best lambda / # features? it's quite possible to
make a model that
Github user coderh commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-45338790
Just a question on the result.
```
implicitPref rank numInterations lambda - rmse
true 30 40 1.0 - 0.5776665087027969
```
Here,
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-45339833
Simple RMSE is not a great metric for this model, because it treats all
errors equally when the model itself does not at all. 1s are much more
important than 0s. The
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42511880
Merged. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42404743
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12388654
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -88,7 +92,27 @@ object MovieLensALS {
val ratings =
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42502536
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42498714
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12388703
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -88,7 +92,27 @@ object MovieLensALS {
val ratings =
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42404750
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12388848
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -121,11 +157,14 @@ object MovieLensALS {
}
/**
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42404869
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42409173
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42498727
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42511606
LGTM. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user techaddict commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12388904
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -88,7 +92,27 @@ object MovieLensALS {
val
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42404871
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14769/
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42463269
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12407647
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -88,7 +92,27 @@ object MovieLensALS {
val ratings =
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42405181
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42405189
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12388902
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -121,11 +157,14 @@ object MovieLensALS {
}
/**
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12388603
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -88,7 +92,27 @@ object MovieLensALS {
val ratings =
Github user techaddict commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42459977
@mengxr changes done :smile: anything else or this good to merge ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user techaddict commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12389144
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -121,11 +157,14 @@ object MovieLensALS {
}
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12407485
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -121,11 +157,23 @@ object MovieLensALS {
}
/**
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42409178
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14770/
---
If your project
Github user techaddict commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42404618
@mengxr Here are few results
```
implicitPref rank numInterations lambda - rmse
true 10 20 1.0 - 0.5985187619423589
true
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42458899
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12407659
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -99,7 +123,18 @@ object MovieLensALS {
val splits =
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42463271
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14777/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42502537
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14793/
---
If your project
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42456355
@techaddict Thanks for running the experiments! The results definitely look
better than `3.x` if we compute RMSE directly for implicit ALS. In the
future, we may
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12388719
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -99,7 +123,18 @@ object MovieLensALS {
val splits =
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/597
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user techaddict commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12362789
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -88,7 +92,11 @@ object MovieLensALS {
val
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12363021
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -88,7 +92,11 @@ object MovieLensALS {
val ratings =
Github user techaddict commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12363376
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -99,7 +107,12 @@ object MovieLensALS {
val
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12362954
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -99,7 +107,12 @@ object MovieLensALS {
val splits =
Github user techaddict commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12362841
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -99,7 +107,12 @@ object MovieLensALS {
val
Github user techaddict commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12362778
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -88,7 +92,11 @@ object MovieLensALS {
val
Github user techaddict commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12363364
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -99,7 +107,12 @@ object MovieLensALS {
val
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42273467
+1 on @srowen 's suggestion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42273875
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user techaddict commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42273835
@mengxr now good ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42273881
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42274037
@techaddict For training, we should keep the `r - 2.5`s, which indicate
confidence. For evaluation, we could either use `if (r 2.5) 1.0 else 0.0` or
weighted RMSE
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42275971
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42275972
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14708/
---
If your project
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42278530
`ratings` is correct. `test` also need to check `implicitPrefs`. If
`implicitPrefs`, predictions should map to `if (pred 1.0) 1.0 else if (pred
0.0) 0.0 else pred`.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42280860
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42280865
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42284625
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14713/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42284620
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12334467
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -62,6 +63,9 @@ object MovieLensALS {
opt[Unit](kryo)
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12334516
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -88,7 +92,11 @@ object MovieLensALS {
val ratings =
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12334664
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -99,7 +107,12 @@ object MovieLensALS {
val splits =
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12334735
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -121,11 +135,17 @@ object MovieLensALS {
}
/**
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12334771
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -121,11 +135,17 @@ object MovieLensALS {
}
/**
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/597#discussion_r12334839
--- Diff:
examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
@@ -121,11 +135,17 @@ object MovieLensALS {
}
/**
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42328612
@techaddict Could you try running this example on ml-1m:
http://grouplens.org/datasets/movielens/ with `--implicitPrefs`? Try different
combinations of rank and number of
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42256964
MovieLens ratings are on a scale of 1-5:
~~~
5: Must see
4: Will enjoy
3: It's okay
2: Fairly bad
1: Awful
~~~
So we should not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42261805
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42261808
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42264123
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14695/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42264121
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42269236
Can I make a tiny suggestion to map from ratings to weights with something
like rating - 2.5 instead of rating - 3? So that 3 becomes a small positive
value like 0.5?
Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42125746
It is true that implicit prefs predict 0/1 (ie a preference matrix rather
than a rating matrix), but the ratings are taken as confidence levels
indicating preference (or
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42125991
On this note, recall there was a change a while back to handle the case of
negative confidence levels. 0 still means don't know and positive values mean
confident that the
Github user techaddict commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-42003703
Mapping rating in case of ImplicitPref to `{r=0 -- 0`, `r0 -- 1}`,
`Rating(fields(0).toInt, fields(1).toInt, fields(2).toDouble)` to
`Rating(fields(0).toInt,
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-41770652
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user techaddict opened a pull request:
https://github.com/apache/spark/pull/597
SPARK-1668: Add implicit preference as an option to examples/MovieLensALS
Add --implicitPrefs as an command-line option to the example app
MovieLensALS under examples/
You can merge this pull
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-41770645
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-41773593
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-41773594
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14584/
---
If your project
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/597#issuecomment-41827788
@techaddict Thanks for working on this JIRA. You also need to change the
evaluation code. Implicit ALS predicts 0/1 instead of the original rating. So
you need some
80 matches
Mail list logo
