Solr and Elasticsearch have near realtime index updates and the indicators can be batch calculated and updated on the order of every few minutes. Spark has streaming for handling new user actions and all this is scalable.
Too me the first step is just being able to query and get recs from a service. Ultimately this needs to support the new ideas we’ve been talking about or it will just be a “me too” project. These innovations do not exist in any recommender that I know of. Those new things include: * blending collaborative filtering, metadata, and content both for training and in the recs returning query * handle multiple actions to recommend the primary action * noise injected into recs to widen the exposure of items and as a form of self-tuning I was thinking to use Play + AngularJS to create a minimal UI and implement the REST + JSON API. On Sep 9, 2014, at 7:45 AM, Saikat Kanjilal <[email protected]> wrote: Maybe we can use spark to do the updates in real time to the recs Sent from my iPhone > On Sep 9, 2014, at 7:37 AM, "Peng Zhang" <[email protected]> wrote: > > That will be a great feature. > > Currently if the offline brach job will run hours to update the recs. Can > this api update recs in realtime? i.e. can we update the recs for a user > based on her last few behaviors 5 minutes ago? > > >> On Sep 9, 2014, at 10:28 PM, Pat Ferrel <[email protected]> wrote: >> >> Now that we have the basis of several significant improvements to Mahout’s >> recommender it seems like we need to go the last step and provide a service. >> Without this it is left to the user to do a lot of integration making the >> current next gen somewhat incomplete. >> >> Using the Hadoop mapreduce code you can get all recs for all people using >> collaborative filtering data or you can use the in-memory single machine >> recommender if you have a small dataset. >> >> The next generation would require Solr or Elasticsearch so why not go the >> extra step and provide a recommender API on top? At very least it would give >> users a single machine API they can call, analogous to the in-memory >> recommender of Mahout 0.9. But it would also be indefinitely scalable. >> >> Is anyone interested in discussing this here? >
