Behavior is a much better basis for recommendation than user ratings. Raters are self-selected. If you can track whether someone actually watches the video, that is gold data.
There is a meme in machine learning that signal drives out noise as data increases, so you can get away with simpler algorithms as you get more data. On Sun, Aug 26, 2012 at 8:38 AM, Sebastian Schelter <[email protected]> wrote: > If you have temporal information, you should use these to split the data. > Try to predict later interactions from older ones. > > Am 26.08.2012 17:04 schrieb >> >> It's the same idea, but yes you'd have to re-implement it for Hadoop. >> >> Randomly select a subset of users. Identify a small number of >> most-preferred items for that user -- perhaps the video(s) watched >> most often. Hold these data points out as a test set. Run your process >> on all the rest. >> >> Make recommendations for the selected users. You then just see how >> many in the list were among the test data you held out. The percentage >> of recs that were in the test list is precision, and the percent of >> the test list in the recs is recall. >> >> Precision and recall are not good tests, but among the only ones you >> can carry out in the lab. Slightly better are variations on these two >> metrics, like F1 measure and normalized discounted cumulative gain. >> Also look up mean average precision. >> >> On Sun, Aug 26, 2012 at 10:47 AM, Jonathan Hodges <[email protected]> >> wrote: >> > Hi, >> > >> > We have been tasked with producing video recommendations for our users. >> We >> > get about 100 million video views per month and track users and the >> videos >> > they watch, but currently we don’t collect rating value or preference. >> > Later we plan on using implicit data like percentage of video watched to >> > surmise preferences but for the first release we are stuck with Boolean >> > viewing data. To that end we started by using Mahout’s distributed >> > RecommenderJob with LoglikelihoodSimilarity algorithm to generate 50 >> video >> > recommendations for each user. We would like to gauge how well we are >> doing >> > by offline measuring precision and recall of these recommendations. We >> know >> > we should divide the viewing data into training and test data, but not >> real >> > sure what steps to take next. For the non-distributed approach we would >> > leverage IRStatistics to get the precision and recall values, but it >> seems >> > there isn’t as simple a solution within the Mahout framework for the >> Hadoop >> > based calculations. >> > >> > Can someone please share/suggest their techniques for evaluating >> > recommendation accuracy with Mahout’s Hadoop-based distributed >> algorithms? >> > >> > Thanks in advance, >> > >> > Jonathan >> -- Lance Norskog [email protected]
