I would not quite say it that way. The notion of "good" itself is not random: the "good" items are all of the ones that the user is associated to. The issue is that they're all equally good, so you have no basis on which to choose some as the most relevant for test purposes. You pick randomly.
You can also define the test/training set manually if you want, yes. If you mean, are there other metrics, I'd look at normalized discounted cumulative gain (nDCG), also implemented. It provides perhaps a slightly more useful metric, since it scores results higher for putting relevant results higher. Sean On Tue, Apr 10, 2012 at 7:54 AM, Janina <[email protected]> wrote: > Hi all, > > currently I am implementing a recommender based on a boolean pref data > model. I have to evaluate this recommender and I have tried to use the > precision and recall measurements Mahout provides. In Mahout-in-Action it > says: > > "There isn’t even a notion of relative preference on which to select > a subset of good items. The best the test can do is randomly select some > preferred items as the good ones." > > Does this mean the notion of what is good is really just random? Do you > maybe have another idea what I can use to evaluate my recommender? Or do I > have to manually define what is good to test the recommender? This would be > very hard because of the large data set I use... > > Thanks a lot and greetings, > Janina
