Another thought here is that in the past, my own designs have essentially done this, but a bit more crudely. We "decayed" old data in the sense that we kept all the data we could eat in the time allowed for daily processing. We "down-weighted" short visits by putting a threshold on the viewing time and ignored all views shorter than the threshold. This heavy hammer approach is actually kind of hard to beat largely because it is very hard to find much gain in data that you already know that you don't particularly like to talk about (which is why you are down-weighting it). The exception to the all or nothing approach was in sampling of data for popular items. There we looked back as far as necessary to get enough data for each item up to our limit. That focused popular items on the recent past, but used longer-term averages for the fringe or long-tail items.
These mechanisms worked pretty well for us and if it were to do over again, I would definitely not spend the time implementing a fancy weighting scheme without evidence that it would actually help things. Even just figuring out how to parametrize, measure and optimize the weighting scheme is a big undertaking which would make me even less likely to consider it as an early design option. On Wed, Jul 21, 2010 at 5:02 PM, Ted Dunning <[email protected]> wrote: > > This is, roughly, a reasonable thing to do. > > If you want to maintain the fiction of counts a little bit more closely, > you might consider just having counts decay over time and having short > visits only give partial credit. > > > On Wed, Jul 21, 2010 at 3:54 PM, Dave Williford > <[email protected]>wrote: > >> We are currently using LogLiklihoodSimilarity to create item >> recommendations >> based on page visits on our web site. We would like to influence the >> generated recommendations for such factors as age of visit (weigh more >> recent visits more heavily), duration of page view (longer is better), >> same >> visit is better than cross-visit (things looked at on the same day are >> more >> related than items looked at by a given user across visits). >> >> I am considering introducing scores for each user/page data point. This >> would essentially replace the integer calculations (which are based on >> summing total data points for each item, total items, and the intersection >> of item A with item B) with real numbers. We could always round the sums >> to >> integers before sending through the loglikelihood calculation although I >> am >> not sure this is necessary. >> >> Note these score are not the same conceptually as preferences so I don't >> think switching to a preference based algorithm would give satisfactory >> results. >> >> I am very new to all of this and am wondering if I am completely off base >> or >> if this seems like a valid approach. Any input is much appreciated. >> > >
