Thanks so much that sped it up incredible amounts to the same speed as when i just use the .txt in memory. However, it's still taking a full minute to get a recommendation for a user.
Are there any ways to speed it up more? On Tue, Jul 24, 2012 at 4:49 PM, Sean Owen <[email protected]> wrote: > Unless your data set is tiny (thousands of users / items), you can't really > run straight off a database. It is far too data intensive. Real-time always > means "in memory" to me. > > Look at the ReloadFromJDBCDataModel wrapper, which will cache the DB data > in memory. This should be orders of magnitude faster. Of course, you have > to have enough memory. But the scale you describe should comfortably fit on > a normal server machine. > > Sean > > On Tue, Jul 24, 2012 at 8:46 PM, Jonathan Nassau > <[email protected]>wrote: > > > I’m writing an item based recommender and am getting extreme bottlenecks > on > > my speed during the “doGetCandidateItems” I’m open to *any *suggestions > on > > how to improve the speed because right now it’s taking over an hour which > > makes no sense. Even if I use a .txt as my dataModel is it very slow. > > > > The table has all the preferences (1000) for every single user (115000) > and > > there are an average of only (50) prefs for user. > > > > The full table will obviously have more than this but that’s the very > small > > subset I’m using. > > > > > > > > I want to recommend items and use RecommendedBecause() on each item to > give > > some detail about each recommendation but with the speed like this I > can’t > > even do that. > > > > > > Thanks so much code is below, > > > > Jonathan > > > > > > > > > > > > MySQLJDBCDataModel dataModel=*null*; > > > > *try* { > > > > Class.*forName*( > > "net.sourceforge.jtds.jdbc.Driver"); > > > > > > net.sourceforge.jtds.jdbcx.JtdsDataSource ds = > > *new*net.sourceforge.jtds.jdbcx.JtdsDataSource(); > > > > ds.setServerName("xxxx"); > > > > ds.setDatabaseName("xxxx"); > > > > ds.setUser("xxxx"); > > > > ds.setPassword(*password*); > > > > ds.setDomain("xxxx"); > > > > > > > > > > > > > > > > dataModel = *new* MySQLJDBCDataModel( > > > > ds, > "test_tbl", > > "user_id", > > > > "item_id", > > "preference",*null*); > > > > > > > > } *catch*(Exception e) > {System.*out*.println("can't > > connect");} > > > > > > > > > > ItemSimilarity similarity = > > *new*FileItemSimilarity( > > *new* File("output/part-r-00000")); > > > > ItemBasedRecommender recommender = > > *new*GenericItemBasedRecommender(dataModel, similarity); > > > > Recommender cachingRecommender = > > *new*CachingRecommender(recommender); > > > > List<RecommendedItem> > > uRec=cachingRecommender.recommend(userid,10); > > > > System.*out*.print("Recommendations:"+ > > uRec); > > > > } > > >
