I lowered the precision @ number to 2, without results. I don't think it's a problem of relevance threshold since it never calls my recommender, so it can't compute any relevant or not relevant item. I'm using Boolean preference data (an item was bought or not bought by a customer)
-----Messaggio originale----- Da: Sean Owen [mailto:[email protected]] Inviato: mercoledì 26 agosto 2009 16.15 A: [email protected] Oggetto: Re: Problems with evaluator. I can tell you what the symptom likely means, but yeah it is hard to help debug a port of an older version of the code, remotely. It basically means you didn't get enough information to compute precision or recall. - Is your evaluation percentage too low? Might increase it if it's close to 0. If it's very low there's a chance you will filter out all users and evaluate nothing. - Do your users have few preferences? If you're computing "precision @ 3" for instance then the evaluator will only bother considering evaluating for users with at least 2*3 = 6 preferences. If it has fewer preferences, the result is probably not meaningful. - Maybe set a lower relevance threshold? If it's too high, and no items are 'relevant', then it can't figure precision or recall, which depend on distinguishing 'relevant' and 'non-relevant' results On Wed, Aug 26, 2009 at 2:55 PM, Claudia Grieco<[email protected]> wrote: > Hi, > > I'm using a version of mahout prior to MAHOUT_150, since the changes after > that version were too big. > > I'm having a problem with an evaluation class > (GenericRecommenderIRStatsEvaluator): it always returns precision and recall > as NaN values. I'm using a c# port of mahout so I can't run a detailed debug > and understand what is wrong, but from some experiments I understood it > never calls my recommender (it obtains data of each user, calculate their > neighborhood and that's all). > > Here is some code I'm using: > > GenericRecommenderIRStatsEvaluator evaluator = new > GenericRecommenderIRStatsEvaluator(); > > MyDataSource source = new MyDataSource(); //a jdbc data source > > ConnectionPoolDataSource poolSource = new ConnectionPoolDataSource(source); > > DataModel model = new MyDataModel(poolSource, "iscrizioni", "studentID", > "ofalID", "score"); > > > > //builder is a RecommenderBuilder that returns my > UserRecommender > > IRStatistics statistics = evaluator.evaluate(builder, model, null, 5, > Double.NegativeInfinity, 0.5); > > string results = "Precision:" + statistics.getPrecision() + " Recall" + > statistics.getRecall(); > > Console.WriteLine("results:" + results); > > > > > > My recommender has always worked, so I don't think it's its fault. > > Thanks in advance > >
