I can tell you what the symptom likely means, but yeah it is hard to help debug a port of an older version of the code, remotely. It basically means you didn't get enough information to compute precision or recall.
- Is your evaluation percentage too low? Might increase it if it's close to 0. If it's very low there's a chance you will filter out all users and evaluate nothing. - Do your users have few preferences? If you're computing "precision @ 3" for instance then the evaluator will only bother considering evaluating for users with at least 2*3 = 6 preferences. If it has fewer preferences, the result is probably not meaningful. - Maybe set a lower relevance threshold? If it's too high, and no items are 'relevant', then it can't figure precision or recall, which depend on distinguishing 'relevant' and 'non-relevant' results On Wed, Aug 26, 2009 at 2:55 PM, Claudia Grieco<[email protected]> wrote: > Hi, > > I'm using a version of mahout prior to MAHOUT_150, since the changes after > that version were too big. > > I'm having a problem with an evaluation class > (GenericRecommenderIRStatsEvaluator): it always returns precision and recall > as NaN values. I'm using a c# port of mahout so I can't run a detailed debug > and understand what is wrong, but from some experiments I understood it > never calls my recommender (it obtains data of each user, calculate their > neighborhood and that's all). > > Here is some code I'm using: > > GenericRecommenderIRStatsEvaluator evaluator = new > GenericRecommenderIRStatsEvaluator(); > > MyDataSource source = new MyDataSource(); //a jdbc data source > > ConnectionPoolDataSource poolSource = new ConnectionPoolDataSource(source); > > DataModel model = new MyDataModel(poolSource, "iscrizioni", "studentID", > "ofalID", "score"); > > > > //builder is a RecommenderBuilder that returns my > UserRecommender > > IRStatistics statistics = evaluator.evaluate(builder, model, null, 5, > Double.NegativeInfinity, 0.5); > > string results = "Precision:" + statistics.getPrecision() + " Recall" + > statistics.getRecall(); > > Console.WriteLine("results:" + results); > > > > > > My recommender has always worked, so I don't think it's its fault. > > Thanks in advance > >
