Unless your data set is tiny (thousands of users / items), you can't really run straight off a database. It is far too data intensive. Real-time always means "in memory" to me.
Look at the ReloadFromJDBCDataModel wrapper, which will cache the DB data in memory. This should be orders of magnitude faster. Of course, you have to have enough memory. But the scale you describe should comfortably fit on a normal server machine. Sean On Tue, Jul 24, 2012 at 8:46 PM, Jonathan Nassau <[email protected]>wrote: > I’m writing an item based recommender and am getting extreme bottlenecks on > my speed during the “doGetCandidateItems” I’m open to *any *suggestions on > how to improve the speed because right now it’s taking over an hour which > makes no sense. Even if I use a .txt as my dataModel is it very slow. > > The table has all the preferences (1000) for every single user (115000) and > there are an average of only (50) prefs for user. > > The full table will obviously have more than this but that’s the very small > subset I’m using. > > > > I want to recommend items and use RecommendedBecause() on each item to give > some detail about each recommendation but with the speed like this I can’t > even do that. > > > Thanks so much code is below, > > Jonathan > > > > > > MySQLJDBCDataModel dataModel=*null*; > > *try* { > > Class.*forName*( > "net.sourceforge.jtds.jdbc.Driver"); > > > net.sourceforge.jtds.jdbcx.JtdsDataSource ds = > *new*net.sourceforge.jtds.jdbcx.JtdsDataSource(); > > ds.setServerName("xxxx"); > > ds.setDatabaseName("xxxx"); > > ds.setUser("xxxx"); > > ds.setPassword(*password*); > > ds.setDomain("xxxx"); > > > > > > > > dataModel = *new* MySQLJDBCDataModel( > > ds, "test_tbl", > "user_id", > > "item_id", > "preference",*null*); > > > > } *catch*(Exception e) {System.*out*.println("can't > connect");} > > > > > ItemSimilarity similarity = > *new*FileItemSimilarity( > *new* File("output/part-r-00000")); > > ItemBasedRecommender recommender = > *new*GenericItemBasedRecommender(dataModel, similarity); > > Recommender cachingRecommender = > *new*CachingRecommender(recommender); > > List<RecommendedItem> > uRec=cachingRecommender.recommend(userid,10); > > System.*out*.print("Recommendations:"+ > uRec); > > } >
