Without a doubt. The business/sociology question that I'll need to answer is if users are shown poor recommendations are more/less/or as likely to provide there own preferences, or if it is better to show more accurate recommendations and have them provide their own interactions based on another system outside of the recommendations themselves (e.g. gamification).
Collaborative filtering can be a very powerful tool, but when the data set is built from user interactions I try to be careful as users can be pretty finicky. For what it's worth I've created my own class extending AbstractItemSimilarity and have the mean difference down to ~0.4. Now I need to do some analysis on how sparse the recommendations are now that I'm excluding some of the data that wasn't what I'll call well-connected. Nick On Wed, Jan 4, 2012 at 10:34 AM, Ted Dunning <[email protected]> wrote: > This may not be a great idea. > > Remember that a real recommend exists in a closed loop. As such, > incorporating iffy recommendations is often a good thing since it helps the > system explore the space. Often I find that the most important thing I can > do is help a weak recommender explore more by dithering the results and > adding anti-flood provisions. Bigger/broader data wins is the moral. > > On Wed, Jan 4, 2012 at 6:52 AM, Nick Jordan <[email protected]> wrote: > > > Thanks for the feedback. In my particular scenario, I'd rather that the > > Recommender only return recommendations for items where the expected > margin > > of error were smaller, even if that meant for a specific set of users no > > recommendations were made or that a specific set of items could never be > > recommended. Maybe what I'm describing is my own home grown Recommender, > > which is fine but I just want to confirm. > > > > It also appears that evaluator uses estimatePreference in the Recommender > > to produce it's output and estimatePreference doesn't take a Rescorer > > parameter, so even if I handled this in Rescorer the Evaluator would not > > pick it up as part of its output. Is that also correct? > > > > Nick > > > > On Wed, Jan 4, 2012 at 8:53 AM, Sean Owen <[email protected]> wrote: > > > > > After thinking about it more, I think your theory is right. > > > > > > You really should use more like 90% of your data to train, and 10% to > > test, > > > rather than the other way around. Here it seems fairly clear that the > 10% > > > training test is returning a result that isn't representative of the > real > > > performance. That's how I'd really "fix" this, plain and simple. > > > > > > Sean > > > > > > On Wed, Jan 4, 2012 at 11:42 AM, Nick Jordan <[email protected]> wrote: > > > > > > > Yeah, I'm a little perplexed. By low-rank items I mean items that > > have a > > > > low number of preferences not a low average preference. Basically if > > we > > > > don't have some level of confidence in our ItemSimilarity based on > the > > > fact > > > > that not many people have given a preference good or bad, don't > > recommend > > > > them. To your point though LogLikelihood may already account for > that > > > > making these results even more surprising. > > > > > > > > > > > > > >
