On Fri, 2007-02-23 at 12:09 -0500, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Tres Seaver wrote: > > Roché Compaan wrote: > >>> On Fri, 2007-02-23 at 06:55 -0500, Tres Seaver wrote: > >>>> Roché Compaan wrote: > >>>>> I'm curious, has anybody played around with the idea of caching ZCatalog > >>>>> results and if I submitted a patch to do this would it be excepted? > >>>>> > >>>>> I quickly coded some basic caching of results on a volatile attribute > >>>>> and I was really surprised with the amount of cache hits I got > >>>>> (especially with a Plone site that is a heavy user of the catalog) > >>>> +1. I think using the 'ZCachable' stuff (e.g., adding a RAMCacheManager > >>>> and associating a catalog to it) would be the sanest path here. > >>> Cool idea. I haven't done any coding involving OFS.Cache though. Looking > >>> at it briefly it looks like one can modify the catalog to subclass > >>> OFS.Cacheable and then use the ZCacheable_get, ZCacheable_set and > >>> ZCacheable_invalidate methods to interact with a cache manager. This > >>> needs to be pretty explicit though. Are there any side effects that I > >>> should guard against if the catalog subclasses OFS.Cache? > > > > I don't think so. Here are some random thoughts on the idea: > > > > - The 'searchResults' method must pass its keyword arguments as > > part of the cache key. > > > > - I don't know if there is a reasonable way to do 'mtime' for > > the catalog: we would like to be able to get an mtime cheaply > > for the BTrees (indexes, the 'data' container), but I don't know > > if that is possible. > > > > - The "right" place to do this feels like the 'searchResults' of > > ZCatalog, just before it calls 'self._catalog.searchResults'. > > > > - The CMF's catalog overrides 'searchResults', but calls it at > > the end, so everything there should work.
In my prototype I also wired the caching into searchResults: def searchResults(self, REQUEST=None, used=None, _merge=1, **kw): ... cache_key = None if args: cache_key = self._makeCacheKey(args) result = self._getCachedResult(cache_key) if result: return result return self._cacheResult(cache_key, self.search(args, sort_index, reverse, sort_limit, _merge)) > > Hmm, on further thought: > > - It isn't safe to stash persistent objects in the RAM Cache manager, > because they can't be used safely from another database connection. But the lazy map of brains isn't persistent? > > - The result set you get back from a query is a "lazy", which will > be consumed by each client: no two clients will see the same > thing. I don't follow. The Lazy will contain a set of document ids that will be the same for all clients, not? I got satisfactory results by storing results in a volatile attribute (and they are not shared by clients). I'm still curious to see what can be achieved with ZCacheable to extend the lifetime of the cache. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )