[
https://issues.apache.org/jira/browse/SPARK-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200396#comment-14200396
]
Sean Owen commented on SPARK-4231:
----------------------------------
So this method basically computes where each test item would rank if you asked
for a list of recommendations that ranks every single item. It's not
necessarily efficient, but is simple. The reason I did it that way was to avoid
recreating a lot of the recommender ranking logic. I don't think one has to
define MAP this way -- I effectively averaged over all k to the # of items.
Yes I found the straightforward definition hard to implement at scale. I ended
up opting to compute an approximation of AUC for recommender eval in this next
version I'm working on:
https://github.com/OryxProject/oryx/blob/master/oryx-ml-mllib/src/main/java/com/cloudera/oryx/ml/mllib/als/AUC.java#L106
Sorry for the hard-to-read Java 7; going to redo this in Java 8 soon.
Basically you're just sampling random relevant/not-relevant pairs and comparing
their scores. You might consider that.
I dunno if it's worth bothering with a toy implementation in the examples. The
example is already just to show Spark really not ALS.
> Add RankingMetrics to examples.MovieLensALS
> -------------------------------------------
>
> Key: SPARK-4231
> URL: https://issues.apache.org/jira/browse/SPARK-4231
> Project: Spark
> Issue Type: Improvement
> Components: Examples
> Affects Versions: 1.2.0
> Reporter: Debasish Das
> Fix For: 1.2.0
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> examples.MovieLensALS computes RMSE for movielens dataset but after addition
> of RankingMetrics and enhancements to ALS, it is critical to look at not only
> the RMSE but also measures like prec@k and MAP.
> In this JIRA we added RMSE and MAP computation for examples.MovieLensALS and
> also added a flag that takes an input whether user/product recommendation is
> being validated.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
