@Sebastian What do u mean with a standalone recommender? A simple offline java main program?
Am 17.04.2014 um 11:41 schrieb Sebastian Schelter <[email protected]>: > Could you take the output of the precomputation, feed it into a standalone > recommender and test it there? > > > On 04/17/2014 11:37 AM, Najum Ali wrote: >> @sebastian >> >>> Are you sure that the precomputation is done only once and not in every >>> request? >> Yes, a @Bean annotated Object is in Spring per default a singleton instance. >> I also just tested it out using a System.out.println() >> Here is my log: >> >> System.out.println("----> precomputation done!“ is called before returning >> the >> GenericItemSimilarity. >> >> The first two recommendations are Item-based -> pearson similarity >> The thrid and 4th log are also item-based using pre computed similarity >> The last log is the userbased recommender using pearson >> >> Look at the huge time difference! >> >> Am 17.04.2014 um 11:23 schrieb Sebastian Schelter <[email protected] >> <mailto:[email protected]>>: >> >>> Najum, >>> >>> this is really strange, feeding an ItemBased Recommender with precomputed >>> similarities should give you superfast recommendations. >>> >>> Are you sure that the precomputation is done only once and not in every >>> request? >>> >>> --sebastian >>> >>> On 04/17/2014 11:17 AM, Najum Ali wrote: >>>> Hi guys, >>>> >>>> I have created a precomputed item-item-similarity collection for a >>>> GenericItemBasedRecommender. >>>> Using the 1M MovieLens data, my item-based recommender is only 40-50% >>>> faster >>>> than without precomputation (like 589.5ms instead 1222.9ms). >>>> But the user-based recommender instead is really fast, it´s like 24.2ms? >>>> How can >>>> this happen? >>>> >>>> Here are more details to my Implementation: >>>> >>>> CSV File: 1M pref, 6040 Users, 3706 Items >>>> >>>> For my Implementation I´m using screenshots, because having the good >>>> highlighting. >>>> My Recommender runs inside a Webserver (Jetty) using Spring 4 and Java8. I >>>> receive Recommendations as Webservice (JSON). >>>> >>>> For DataModel, I´m using FileDataModel. >>>> >>>> >>>> This code below creates me a precomputed ItemSimilarity when I start the >>>> Webserver and the property isItemPreComputationEnabled is set to true: >>>> >>>> >>>> For time measuring I´m using AOP. I´m measuring the whole time from >>>> entering my >>>> Controller to sending the response. >>>> based on System.nanoTime(); and getting the diff. It´s the same time >>>> measure for >>>> user based. >>>> >>>> I haved tried to cache the recommender and the similarity with no big >>>> difference. I also tried to use CandidateItemsStrategy and >>>> MostSimilarItemsCandidateItemsStrategy, but also no performance boost. >>>> >>>> public RecommenderBuilder createRecommenderBuilder(ItemSimilarity >>>> similarity) >>>> throws TasteException { >>>> final int numberOfUsers = dataModel.getNumUsers(); >>>> final int numberOfItems = dataModel.getNumItems(); >>>> CandidateItemsStrategy candidateItemsStrategy = new >>>> SamplingCandidateItemsStrategy(numberOfUsers,numberOfItems); >>>> MostSimilarItemsCandidateItemsStrategy mostSimilarStrategy = new >>>> SamplingCandidateItemsStrategy(numberOfUsers,numberOfItems); >>>> return model -> new GenericItemBasedRecommender(model, >>>> similarity,candidateItemsStrategy,mostSimilarStrategy); >>>> } >>>> >>>> I dont know why item-based is taking so much longer then user-based. >>>> User-based >>>> is like fast as hell. I even tried a DataSet using 100k Prefs, and >>>> 10Million >>>> (Movielens). Everytime the user-based is soo much faster for any >>>> similarity. >>>> >>>> Hope you anyone can help me to understand this. Maybe I´m doing something >>>> wrong. >>>> >>>> Thanks!! :))
