That would be great: Specifically if that is some kind of real usage data, and the results are evaluated against a -without decay- baseline, via A/B tests measuring the increase in conversion.
Best Gokhan On Wed, Nov 20, 2013 at 2:28 PM, Cassio Melo <melo.cas...@gmail.com> wrote: > Hi guys, thanks for sharing your experiences on this subject, really > appreciated. To summarize the discussion: > > - The decay of old preference values might loose important historical data > in cases where the user has no recent activity (Gokhan) > - When using decay (or truncate preferences), the precision of rating > prediction may be lower (Pat, Gokhan, Ted) but it might increase conversion > rates (Gokhan, Pat) since it reflects recent user intent. > - Tweaking the score estimation may be a better approach (Gokhan) > > I'm doing some experiments with e-commerce data, I'll post the results > later. > > Best regards, > Cassio > > > On Fri, Nov 8, 2013 at 5:08 PM, Pat Ferrel <pat.fer...@gmail.com> wrote: > > > > I think the intuition here is, when making an item neighborhood base > > > recommendation, to penalize the contribution of the items that the user > > has > > > rated a long time ago. I didn't test this in a production recommender > > > system, but I believe this might result in recommendation lists with > > better > > > conversion rates in certain use cases. > > > > It’s only one data point but it was a real ecom recommender with real > user > > data. We did not come to the conclusion above, though there is some truth > > in it. > > > > There are two phenomena at play, similarity of users and items, and > recent > > user intent. Similarity of users decays very slowly if at all. The fact > > that you and I bought an iPhone 1 makes us similar even though the > iPhone 1 > > is no longer for sale. However you don’t really want to rely on user > > activity that old to judge recent shopping intent. Mahout conflates these > > unfortunately. > > > > Back to the canonical R = [B’B]H; [B’B] is actually calculated using some > > similarity metric like log-likihood and RowSimilarityJob. > > B = preference matrix; user = row, item = column, value = strength > perhaps > > 1 for a purchase. > > H = user history of preferences in columns, rows = items > > > > If you did nothing to decay preferences B’=H > > > > If you truncate to use only recent preferences in H then B’ != H > > > > Out of the box Mahout requires B’=H, and we got significantly lower > > precision scores by decaying BOTH B and H. Our conclusion was that this > was > > not really a good idea given our data. > > > > If you truncate user preferences to some number of the most recent in H > > you probably get a lower precision score (as Ted mentions) but our > > intuition was that the recommendations reflect the most recent user > intent. > > Unfortunately we haven’t A/B tested this conclusion but the candidate for > > best recommender was using most recent prefs in H and all prefs in B. > > > > > On Nov 7, 2013, at 11:36 PM, Gokhan Capan <gkhn...@gmail.com> wrote: > > > > On Fri, Nov 8, 2013 at 6:24 AM, Ted Dunning <ted.dunn...@gmail.com> > wrote: > > > > > On Thu, Nov 7, 2013 at 12:50 AM, Gokhan Capan <gkhn...@gmail.com> > wrote: > > > > > >> This particular approach is discussed, and proven to increase the > > > accuracy > > >> in "Collaborative filtering with Temporal Dynamics" by Yehuda Koren. > The > > >> decay function is parameterized per user, keeping track of how > > consistent > > >> the user behavior is. > > >> > > > > > > Note that user-level temporal dynamics does not actually improve the > > > accuracy of ranking. It improves the accuracy of ratings. > > > > > > Yes, the accuracy of rating prediction. > > > > Since > > > recommendation quality is primarily a precision@20 sort of activity, > > > improving ratings does no good at all. > > > > > > > Item-level temporal dynamics is a different beast. > > > > > > > I think the intuition here is, when making an item neighborhood base > > recommendation, to penalize the contribution of the items that the user > has > > rated a long time ago. I didn't test this in a production recommender > > system, but I believe this might result in recommendation lists with > better > > conversion rates in certain use cases. > > > > Best > > > > >
