There is as yet no Mahout 0.6 release. The Mahout trunk is "post 0.5" and what we're calling 0.6. You want to use the trunk. It gets a lot of testing and production use, and these days stays pretty solid.
Lance 2011/10/21 WangRamon <[email protected]> > > Ok Sebastian, I will try Mahout 0.6 next week, i believe it's from trunk, > right? Have a nice day/weekend! Cheers Ramon > > Date: Fri, 21 Oct 2011 09:06:50 +0200 > > From: [email protected] > > To: [email protected] > > Subject: Re: Recommend result contains item which user has already given > preference, is that correct? > > > > As I already said multiple times, please use Mahout 0.6. It contains bug > > fixes and performance improvements for this particular job. > > > > --sebastian > > > > On 21.10.2011 09:04, WangRamon wrote: > > > > > > Hi Sebastian I made the following change to resolve the issue in my > local, it's in Mahout 0.5, maybe i were wrong, but the test result is > correct: 1) I add a "int itemIdIndex" property with getter/setter methods in > class PrefAndSimilarityColumnWritable, it will hold the item index for which > the "prefValue" in this class is for. 2) Add > "prefAndSimilarityColumn.setItemIdIndex(key.get());" in class > PartialMultiplyMapper line 51 to set the item index property created in step > 1. 3) In class AggregateAndRecommendReducer, add the following code in line > 147: // item which user has already given preference > > > int itemIdIndex = prefAndSimilarityColumn.getItemIdIndex(); > > > // exclude item user has already given preference > > > simColumn.set(itemIdIndex, Double.NaN); This will make the > specific index value in the sim column as NaN for item that user has already > given preference, then later plus or multiply this vector will also get a > NaN value in that specific item index, so i exclude the items which user has > already shown preference from recommendation. 4) At line 173 of the same > class AggregateAndRecommendReducer, add a check to make the prediction value > as NaN for those items user has given preference: double prediction = > Double.NaN; > > > if (!Double.isNaN(element.get())) { > > > prediction = element.get() / denominators.getQuick(itemIDIndex); > > > } > > > Then, i get the correct recommendation, I have thought it carefully, > but... maybe wrong, glad to hear your idea, and again, thank you very much. > CheersRamon> From: [email protected] > > >> To: [email protected] > > >> Subject: RE: Recommend result contains item which user has already > given preference, is that correct? > > >> Date: Fri, 21 Oct 2011 10:01:12 +0800 > > >> > > >> > > >> Hi Sebastian Unfortunately, i still get the wrong data from the > RecommenderJob after i clean everything, check the following recommend > result part: 49 > [300420:5.0,312611:5.0,428914:5.0,208617:5.0,345206:5.0,411909:5.0,363683:5.0,248872:5.0,93087:5.0,494200:5.0] > Now, look at the input data for user 49, item 312611, 428914, 208617, > 345206, 411909, 363683, 248872 and 494200 are wrong recommendation, nearly > all of them are wrong, I hope i can send you the test data, but it will be > 50M+ in size, can we discuss offline? Thank you very much. 49,409769,4 > > >> 49,98795,4 > > >> 49,262163,1 > > >> 49,66009,4 > > >> 49,414484,2 > > >> 49,405329,3 > > >> 49,312611,1 > > >> 49,336441,4 > > >> 49,136494,5 > > >> 49,345206,3 > > >> 49,479179,1 > > >> 49,318960,4 > > >> 49,52683,3 > > >> 49,270840,3 > > >> 49,264828,1 > > >> 49,222390,4 > > >> 49,456614,5 > > >> 49,436207,5 > > >> 49,306308,2 > > >> 49,391582,5 > > >> 49,494200,4 > > >> 49,423328,3 > > >> 49,112997,3 > > >> 49,229347,5 > > >> 49,474928,3 > > >> 49,349350,1 > > >> 49,208508,3 > > >> 49,314397,2 > > >> 49,14673,2 > > >> 49,496041,4 > > >> 49,301875,4 > > >> 49,234234,1 > > >> 49,325287,3 > > >> 49,35756,5 > > >> 49,365097,4 > > >> 49,13376,4 > > >> 49,333634,2 > > >> 49,283494,5 > > >> 49,208617,3 > > >> 49,245390,1 > > >> 49,221804,2 > > >> 49,347821,3 > > >> 49,138954,5 > > >> 49,164206,5 > > >> 49,72238,1 > > >> 49,356632,1 > > >> 49,452296,3 > > >> 49,182288,5 > > >> 49,499031,5 > > >> 49,150727,4 > > >> 49,240533,5 > > >> 49,326081,4 > > >> 49,220683,2 > > >> 49,196527,2 > > >> 49,177165,3 > > >> 49,411709,5 > > >> 49,360722,3 > > >> 49,466310,1 > > >> 49,160375,2 > > >> 49,137203,5 > > >> 49,32634,4 > > >> 49,62134,5 > > >> 49,96982,5 > > >> 49,196951,1 > > >> 49,304155,5 > > >> 49,406109,4 > > >> 49,244276,5 > > >> 49,189552,1 > > >> 49,442215,3 > > >> 49,268806,2 > > >> 49,364912,2 > > >> 49,410896,5 > > >> 49,450602,5 > > >> 49,151703,1 > > >> 49,248872,4 > > >> 49,21684,1 > > >> 49,41196,1 > > >> 49,26614,2 > > >> 49,369075,5 > > >> 49,321916,1 > > >> 49,325081,1 > > >> 49,329877,4 > > >> 49,344661,4 > > >> 49,8429,3 > > >> 49,69279,1 > > >> 49,143695,1 > > >> 49,229120,2 > > >> 49,26298,4 > > >> 49,54456,1 > > >> 49,75937,4 > > >> 49,87042,3 > > >> 49,345383,5 > > >> 49,363683,4 > > >> 49,128047,3 > > >> 49,234878,5 > > >> 49,428914,3 > > >> 49,353107,2 > > >> 49,266850,4 > > >> 49,421211,3 > > >> 49,265739,4 > > >> 49,303723,1 > > >> 49,244575,4 > > >> 49,303625,4 > > >> 49,350481,5 > > >> 49,63985,4 > > >> 49,207327,3 > > >> 49,397535,1 > > >> 49,300916,5 > > >> 49,358094,4 > > >> 49,314919,5 > > >> 49,309355,5 > > >> 49,403169,5 > > >> 49,90148,4 > > >> 49,224056,4 > > >> 49,359181,2 > > >> 49,341927,5 > > >> 49,436521,4 > > >> 49,480682,4 > > >> 49,315561,3 > > >> 49,218647,5 > > >> 49,245276,2 > > >> 49,93189,1 > > >> 49,204695,4 > > >> 49,498350,5 > > >> 49,155787,3 > > >> 49,112730,3 > > >> 49,416756,2 > > >> 49,411909,4 > > >> 49,253353,2 > > >> 49,196663,5 > > >> 49,40903,3 > > >> 49,51873,2 > > >> 49,66925,3 > > >> > Date: Thu, 20 Oct 2011 18:40:38 +0200 > > >>> From: [email protected] > > >>> To: [email protected] > > >>> Subject: Re: Recommend result contains item which user has already > given preference, is that correct? > > >>> > > >>> To put it simplified: > > >>> > > >>> The vector of recommendations is the sum of the similarity vectors > for > > >>> all preferred items. In each similarity vector for a preferred item > the > > >>> entry for that particular item is set to NaN. > > >>> > > >>> That means that in the recommendation vector the entries for all > > >>> preferred items will be NaN. > > >>> > > >>> It's a neat trick that is unfortunately very hard to see in the code. > > >>> > > >>> --sebastian > > >>> > > >>> On 20.10.2011 18:36, WangRamon wrote: > > >>>> > > >>>> Hi Sebastian > > >>>> "But as the entry for the item itself is set to NaN in its > similarityvector and NaN plus something stays always NaN, the predicted > preferencefor an item that was already preferred is NaN. And the NaN entries > aredropped later." > > >>>> Wait a minute here, i can understand NaN plus something stays always > NaN, but, how do you explain "the predicted preference for an item that was > already preferred is NaN", where do you put the code to check an item that > was already preferred? The only thing about NaN in > SimilarityMatrixRowWrapperMapper is to say two item (A to A) has a > similarity of NaN, am i right? > > >>>> Thanks > > >>>> Ramon > > >>>>> Date: Thu, 20 Oct 2011 17:04:20 +0200 > > >>>>> From: [email protected] > > >>>>> To: [email protected] > > >>>>> Subject: Re: Recommend result contains item which user has already > given preference, is that correct? > > >>>>> > > >>>>> On 20.10.2011 16:57, WangRamon wrote: > > >>>>>> > > >>>>>> Hi Sebastian and Sean > > >>>>>> Thanks for your help. > > >>>>>> > > >>>>>> I re-read the code again (debug seems to be very difficult for me > to setup the environment) and find the line in > SimilarityMatrixRowWrapperMapper, i past it below with the comments: > > >>>>>> /* remove self similarity */ > > >>>>>> similarityMatrixRow.set(key.get(), Double.NaN); > > >>>>>> I think the meanning is to mark the similarity between Item X and > Item X (the identical one) as NaN, but it doesn't exclude Item X from > recommendation, then in AggregateAndRecommendReducer, it uses > simColumn.times(prefValue) as part of the formula to calculate the > preferences for all items that similar to Item i (it could be Item X or some > other item), then return the top 10 (default) for a user. > > >>>>>> During this process, i cannot see any code to exclude an item > which the user has already given preference from recommendation. > > >>>>> > > >>>>> It's a little bit hidden :) For each preferred item, a vector of > all its > > >>>>> similarities is added: > > >>>>> > > >>>>> numerators = numerators == null > > >>>>> ? prefValue == BOOLEAN_PREF_VALUE ? simColumn.clone() : > > >>>>> simColumn.times(prefValue) > > >>>>> : numerators.plus(prefValue == BOOLEAN_PREF_VALUE ? > simColumn > > >>>>> : simColumn.times(prefValue)); > > >>>>> > > >>>>> But as the entry for the item itself is set to NaN in its > similarity > > >>>>> vector and NaN plus something stays always NaN, the predicted > preference > > >>>>> for an item that was already preferred is NaN. And the NaN entries > are > > >>>>> dropped later. > > >>>>> > > >>>>> --sebastian > > >>>>> > > >>>>> > > >>>>>> Correct me if i miss something, thank you guys. > > >>>>>> Cheers Ramon > > >>>>>>> Date: Thu, 20 Oct 2011 13:59:28 +0100 > > >>>>>>> Subject: Re: Recommend result contains item which user has > already given preference, is that correct? > > >>>>>>> From: [email protected] > > >>>>>>> To: [email protected] > > >>>>>>> > > >>>>>>> Ah OK, figured as much. WangRamon does that answer your question > > >>>>>>> and/or can you debug to see if this is happening, not happening > for > > >>>>>>> you in your use case? > > >>>>>>> > > >>>>>>> On Thu, Oct 20, 2011 at 1:42 PM, Sebastian Schelter < > [email protected]> wrote: > > >>>>>>>> It's still included in SimilarityMatrixRowWrapperMapper. We also > have a > > >>>>>>>> unit test that checks whether a user is only recommended unknown > items > > >>>>>>>> which still works. > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > > -- Lance Norskog [email protected]
