Would it be possible to create appropriate abstract methods in appropriate classes (sorry for vagueness, I'm not yet familiar with the code enough to suggest exact places) that would allow dependent classes to list for events triggered/processed by classes they depend on. Here is a concrete example. I like how the FDM re-checks the file and re-loads it. The Recommender depends on data read by FDM. So I would hope I could add Recommender as a FDM event listener. The FDM might have a method such as addListener(FDMListener), abstract beforeReload(...) and abstract afterReload(...). These methods would be called and implemented by Recommender, which could then clear its cache or do nothing.
Thoughs so far? If the above is doable, would it work for other dependency cases in Taste? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: Sean Owen <[email protected]> > To: [email protected] > Sent: Wednesday, January 28, 2009 1:59:48 PM > Subject: Re: Taste: Clearing CachingRecommender > > Yeah, I have really struggled with this piece. The original vision was > to support real-time updates -- recommendations always use all the > latest data. It was quickly clear that was far too slow and wasteful. > So I bolted on caching... and started putting it everywhere. And it > quickly became a bit of a mess to manage correctly expiring cache > entries and so on. > > I ended up, over time, with the reasonable "Refreshable" scheme which > lets components update their state (and caches) after having > efficiently had their downstream dependencies update. > > You've found a part where that kind of breaks down. I suppose the idea > is that updates are driven solely from upstream components. The > Recommender decides when to update and passes that message down. Under > that view, the behavior is correct -- the FileDataModel is welcome to > update itself, and eventually the upstream components will care to > re-read from it. > > But then perhaps this automatic reloading is pointless. It should > happen on demand, perhaps with a limit to avoid re-reading updates too > frequently or something. > > Let me pause and ask for your thoughts? > > On Wed, Jan 28, 2009 at 6:51 PM, Otis Gospodnetic > wrote: > > Hi, > > > > I'm trying out CachingRecommender to see how much it improves performance > > for > me (it does a lot when the cache is all warmed up). > > One thing that I assumed would happen automatically is that the > CachingRecommender will be refreshed when the underlying file (FileDataModel) > is > refreshed. Instead, I do see the FDM re-reading the input file when it > detects > the change, but my recommendations remain cached forever, as I'm not hitting > memory limits. > > > > I've been looking at the code, but can't see a clean way to "listen" to the > FDM and find out when it finished re-reading the file, so I can clear my > CachingRecommender. Is there an existing way this should be done that I'm > just > not seeing in the code? > > > > Ah, I see another thing now. Consider this: > > > > Recommender recommender = new > > BooleanUserGenericUserBasedRecommender(model, > hood, similarity); > > recommender = new CachingRecommender(recommender); > > > > Now if I did somehow find a way to clear the caching recommender, I imagine > I'd want to call one of its clear(...) methods, and this requires a cast back > to > CachingRecommender. Is there a better way? > > > > Thanks, > > Otis > > -- > > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > >
