[ https://issues.apache.org/jira/browse/SPARK-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200396#comment-14200396 ]
Sean Owen commented on SPARK-4231: ---------------------------------- So this method basically computes where each test item would rank if you asked for a list of recommendations that ranks every single item. It's not necessarily efficient, but is simple. The reason I did it that way was to avoid recreating a lot of the recommender ranking logic. I don't think one has to define MAP this way -- I effectively averaged over all k to the # of items. Yes I found the straightforward definition hard to implement at scale. I ended up opting to compute an approximation of AUC for recommender eval in this next version I'm working on: https://github.com/OryxProject/oryx/blob/master/oryx-ml-mllib/src/main/java/com/cloudera/oryx/ml/mllib/als/AUC.java#L106 Sorry for the hard-to-read Java 7; going to redo this in Java 8 soon. Basically you're just sampling random relevant/not-relevant pairs and comparing their scores. You might consider that. I dunno if it's worth bothering with a toy implementation in the examples. The example is already just to show Spark really not ALS. > Add RankingMetrics to examples.MovieLensALS > ------------------------------------------- > > Key: SPARK-4231 > URL: https://issues.apache.org/jira/browse/SPARK-4231 > Project: Spark > Issue Type: Improvement > Components: Examples > Affects Versions: 1.2.0 > Reporter: Debasish Das > Fix For: 1.2.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > examples.MovieLensALS computes RMSE for movielens dataset but after addition > of RankingMetrics and enhancements to ALS, it is critical to look at not only > the RMSE but also measures like prec@k and MAP. > In this JIRA we added RMSE and MAP computation for examples.MovieLensALS and > also added a flag that takes an input whether user/product recommendation is > being validated. > -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org