Hi Lance,
When evaluating a recommender you should split your dataset in a
training and test part. Only data from the training part should be
included in your DataModel and you only measure the accuracy of
predicting ratings that are included in the test part (which is not
known by your recommender). If you structure things this way, the
current implementation should work fine for you.
--sebastian
On 21.10.2010 04:56, Lance Norskog wrote:
Since this is Recommender day, here is another kvetch:
The recommender implementations with algorithms all do this in
Recommender.estimatePreference():
public float estimatePreference(long userID, long itemID) throws
TasteException {
DataModel model = getDataModel();
Float actualPref = model.getPreferenceValue(userID, itemID);
if (actualPref != null) {
return actualPref;
}
return doEstimatePreference(userID, itemID);
}
Meaning: "if I told you something, just parrot it back to me."
Otherwise, make a guess.
I am doing head-to-head comparisons of the dataModel preferences v.s.
the Recommender. This code makes it impossible to directly compare
what the recommender thinks v.s. the actual preference. If I wanted to
know what I told it, I already know that. I want to know what the
recommender thinks.
If this design decision is something y'all have argued about and
settled on, never mind. If it is just something that seemed like a
good idea at the time, can we change the recommenders, and the
Recommender "contract", to always use their own algorithm?
