Yes, you don't want to use the database directly. A relational database will never be fast enough.
You want to use RefreshFromJDBCDataModel to load the data into memory periodically. This is really what I was referring to. On Mon, May 14, 2012 at 1:49 PM, Nikolaos Romanos Katsipoulakis <[email protected]> wrote: > Thank you for your immediate response. Also, I have one more question. When > I use data from a Mysql database, my recommender is very slow, about 2 > minutes for a 1 million records dataset. The code of the recommender is > presented below: > > this.dataSource = new MysqlDataSource(); > > this.dataSource.setServerName(serverName); > this.dataSource.setUser(user); > this.dataSource.setPassword(pass); > this.dataSource.setDatabaseName(dbName); > > this.dataModel = new MySQLJDBCDataModel(this.dataSource, this.tableName, > this.userColumn, this.itemColumn, this.prefColumn, > this.timeStampColumn); > > this.similarity = new EuclideanDistanceSimilarity(this.dataModel); > this.neighborhood = new NearestNUserNeighborhood(neighborhoodSize, > this.similarity, this.dataModel); > > this.recommender = new GenericUserBasedRecommender(this.dataModel, > this.neighborhood, this.similarity); > List<RecommendedItem> recommendations = null; > > recommendations = recommender.recommend(user, NumOfRecommendations); > > Why is it so slow? I should mention that i also used > MysqlConnectionPoolDataSource() and the performance remains the same. >
