Right, in that case you can pass any threshold less than 1. This will cause the framework to pick the first "at" items it encounters as relevant. Since there is no way of differentiating the items, this is as good as anything. So if you are evaluating precision at 3, it will pick some 3 items as relevant. Sounds like you already tried passing Double.NEGATIVE_INFINITY, which is fine. If it is still not giving results, my other guesses still stand -- low evaluation percentage? too few preferences per user? I think you need to go in with a debugger; it's hard to guess more from here. I can tell you for what it's worth that I am using the same evaluator (in the main Java code of course) to evaluate results in a similar data set and it appears to work fine for me. There could be some issue in the port, or perhaps there was a bug fix since the port happened? I don't remember a bug fix here in a while but could be missing something.
On Thu, Aug 27, 2009 at 10:42 AM, Claudia Grieco<[email protected]> wrote: > But since I'm using boolean preference data, all the preferences are relevant > (score 1) so they are all erased from the training data: > > for (Preference pref : prefs) { > if (pref.getValue() >= theRelevanceThreshold) { > relevantItems.add(pref.getItem()); > } > } > > -----Messaggio originale----- > Da: Sean Owen [mailto:[email protected]] > Inviato: giovedì 27 agosto 2009 11.29 > A: [email protected] > Oggetto: Re: R: R: R: Problems with evaluator. > > Yes, that is correct. The framework splits the user's preferences into > "relevant" and "not relevant" items. It then takes away the relevant > items, and leaves the non-relevant items in the training data. Then, > it sees how many of those relevant items are recommended back to the > user, to compute precision and recall. > > On Thu, Aug 27, 2009 at 10:24 AM, Claudia Grieco<[email protected]> wrote: >> Notice this part: >> for (Preference pref : prefs2) { >> if (!relevantItems.contains(pref.getItem())) { >> trainingPrefs.add(pref); >> }} >> It adds a preference only if it's NOT in the relevant items > >
