On 05/14/2012 02:54 PM, Sean Owen wrote:
Can you persist a DataModel? sure. The easiest thing is to read/write
a CSV file. Or put the data in a database. The existing
implementations already read such things into memory for you. I am not
sure what you mean about computing the DataModel as a separate
process. The DataModel exists and should be used by both online and
offline processes. No need for two.
Can you persist your recommendations? surely, however you like. Create
them, store them in a file or database. That's pretty up to you.
If your problem is such that you can afford to pre-compute many
recommendations (that is, you can afford to compute recommendations
you may not use, and, it's OK if recs are not up-to-the-second) then
your problem is a lot easier. Just run a process (unrelated to Tomcat)
that computes recs continuously in the background and publishes them
to some store, like a file system or database. Hadoop is a fine fit
for this kind of use case here. While it may be overkill, its
drawbacks (batch-oriented-ness) would not be an issue.
Also look at using CachingRecommender to cache frequent recs, rather
than bother with all this. Maybe that's a solution.
On Mon, May 14, 2012 at 12:44 PM, Nikolaos Romanos Katsipoulakis
<[email protected]> wrote:
Hi to everybody,
I am currently developing a recommendation system, that relies on an
Apache Tomcat Server (6.0.26) and is
triggered by Web-Service Calls (JAX-WS).
In order to improve the performance of the system, I have been asked to
asynchronously call the recommender and save the recommendation
data models in persistent storage (MySQL database), in an incremental
manner. For example, I would have a ComputeRecommendations class that
would create the data model, which would produce the recommendations.
Can I save this class in persistent storage so that, at a later time, if
I need to re-compute the recommendations, I would not need to create my
data-model from the start?
Thank you
Thank you for your immediate response. Also, I have one more question.
When I use data from a Mysql database, my recommender is very slow,
about 2 minutes for a 1 million records dataset. The code of the
recommender is presented below:
this.dataSource = new MysqlDataSource();
this.dataSource.setServerName(serverName);
this.dataSource.setUser(user);
this.dataSource.setPassword(pass);
this.dataSource.setDatabaseName(dbName);
this.dataModel = new MySQLJDBCDataModel(this.dataSource, this.tableName,
this.userColumn, this.itemColumn, this.prefColumn,
this.timeStampColumn);
this.similarity = new EuclideanDistanceSimilarity(this.dataModel);
this.neighborhood = new NearestNUserNeighborhood(neighborhoodSize,
this.similarity, this.dataModel);
this.recommender = new GenericUserBasedRecommender(this.dataModel,
this.neighborhood, this.similarity);
List<RecommendedItem> recommendations = null;
recommendations = recommender.recommend(user, NumOfRecommendations);
Why is it so slow? I should mention that i also used
MysqlConnectionPoolDataSource() and the performance remains the same.
