+ Ankur directly, since I am not sure you are on the dev list.
On Tue, May 20, 2008 at 12:06 PM, Sean Owen <[EMAIL PROTECTED]> wrote: > All of the algorithms assume a world where you have a continuous range > of ratings from users for items. Obviously a binary yes/no rating can > be mapped into that trivially -- 1 and -1 for example. This causes > some issues, most notably for corrletion-based recommenders where the > correlation can be undefined between two items/users in special cases > that arise from this kind of input -- for example if we overlap in > rating 3 items and I voted "yes" for all 3, then no correlation can be > defined. > > Slope one doesn't run into this particular mathematical wrinkle. > > Also, methods like estimatePreference() are not going to give you > estimates that are always 1 or -1. Again, you could map this back onto > 1 / -1 by rounding or something, just something to note. > > So, in general it will be better if you can map whatever input you > have onto a larger range of input. You will feed more information in, > in this way, as well. For example, maybe you call a recent "yes" > rating a +2, and a recent "no" a -2, and others +1 and -1. > > > The part of slope one that parallelizes very well is the computing of > the item-item diffs. No I have not written this yet. > > > I have committed a first cut at a framework for computing > recommendations in parallel for any recommender. Dig in to > org.apache.mahout.cf.taste.impl.hadoop. In general, none of the > existing recommenders can be parallelized, because they generally need > access to all the data to produce any recommendation. > > But, we can take partial advantage of Hadoop by simply parallelizing > the computation of recommendations for many users across multiple > identical recommender instances. Better than nothing. In this > situation, one of the map or reduce phase is trivial. > > That is what I have committed so far and it works, locally. I am in > the middle of figuring out how to write it for real use on a remote > Hadoop cluster, and how I would go about testing that! > > Do we have any test bed available? > > > > On Tue, May 20, 2008 at 7:47 AM, Goel, Ankur <[EMAIL PROTECTED]> wrote: >> I just realized after going through the wikipedia that slope one is >> applicable when you have ratings for the items. >> In my case, I would be simply working with binary data (Item was clicked >> or not-clicked by user) using Tanimoto coefficient to calculate item >> similarity. >> The idea is to capture the simple intuition "What items have been >> visited most along with this item". >> >> >> -----Original Message----- >> From: Goel, Ankur [mailto:[EMAIL PROTECTED] >> Sent: Tuesday, May 20, 2008 2:51 PM >> To: [email protected] >> Subject: RE: Taste on Mahout >> >> >> Hey Sean, >> I actually plan to use slope-one to start with since >> - Its simple and known to work well. >> - Can be parallelized nicely into the Map-Reduce style. >> I also plan to use Tanimoto coefficient for item-item diffs. >> >> Do we have something on slope-one already in Taste as a part of Mahout ? >> >> At the moment I am going through the available documentation on Taste >> and code that's present in Mahout. >> >> Your suggestions would be greatly appreciated. >> >> Thanks >> -Ankur >> >> -----Original Message----- >> From: Sean Owen [mailto:[EMAIL PROTECTED] >> Sent: Tuesday, April 29, 2008 11:09 PM >> To: [email protected]; Goel, Ankur >> Subject: Re: Taste on Mahout >> >> I have some Hadoop code mostly ready to go for Taste. >> >> The first thing to do is let you generate recommendations for all your >> users via Hadoop. Unfortunately none of the recommenders truly >> parallelize in the way that MapReduce needs it to -- you need all data >> to compute any recommendation really -- but you can at least get >> paralellization out of this. You can use the framework to run n >> recommenders, each computing 1/nth of all recommendations. >> >> The next application is specific to slope-one. Computing the item-item >> diffs is exactly the kind of thing that MapReduce is good for, so, >> writing a Hadoop job to do this seems like a no-brainer. >> >> On Tue, Apr 29, 2008 at 11:14 AM, Goel, Ankur <[EMAIL PROTECTED]> >> wrote: >>> Hi Folks, >>> What's the status of hadoopifying Taste on Mahout ? >>> What's been done and what is in progress/pending ? >>> >>> I am looking using a scalable version of Taste for my project. >>> So basically trying to figure out what's already done and where I >>> can pitch in. >>> >>> Thanks >>> -Ankur >>> >> >
