Yeah it should be easy and fine to separate the EJB, web service clients further. Beyond that I think it's mostly driven by what we want to achieve, and it sounds like that is Hadoop-ifying it basically.
So far I think of Hadoop as another client. We could provide a wrapper that lets you run n instances of the same recommender, each computing recommendations for a subset of users. This is a crude parallelization but works for any algorithm or setup, and does provide a way to (p)recompute all your user's recommendations in a nightly batch process for example versus online. (There is also one aspect of SlopeOneRecommender which can be readily Hadoopified, and that is computing diffs. There the recommender depends on Hadoop rather than the other way around. Another story.) I suppose I would like to avoid dismantling the recommender's ability to act as an active entity providing online recommendations, since some people want and need that. Of course, anything that can be done in real-time can support a batch process too, so so far no conflict with Hadoop. If this begins to conflict with other goals I am sure we can find ways to restructure the code to meet both needs. For example: thread-safety comes up. My code is prepared for multi-threaded access, but when accessed by a Hadoop job it's only single-threaded. The thread-safeness is unnecessary but doesn't hurt except to the extent it is overhead, but, nothing like a crticial block involves synchronization so I really don't imagine this conflicts with any other goal so far. Sean On Fri, Apr 18, 2008 at 5:37 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote: > I think, for Taste, we need to start figuring out how we are going to > distribute the workload, etc. My gut says the webapp stuff and the core > recommender stuff, etc. should be moved to a "contrib" area, but not sure > just yet on that. That is, not sure how the EJB, web, stuff fits into the > core of Mahout. Will know better once we have the code committed and we can > get several eyes looking at it. > > Of course, perhaps, if and when Mahout is a TLP, Taste will be a complete > self-contained sub project. > > -Grant > > > > On Apr 18, 2008, at 5:07 PM, Sean Owen wrote: > > > > Thread safe, I think, meaning I might have two users making a request > > to a web app, which are handled in two threads, simultaneously > > accessing a Recommender. Stuff like caches need to make sure they > > don't blow up if two people hit it at once. > > > > On Fri, Apr 18, 2008 at 5:02 PM, Ted Dunning <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > Thread safe? Or reentrant? > > > > > > > >
