On Fri, 2007-02-23 at 12:09 -0500, Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> Tres Seaver wrote:
> > Roché Compaan wrote:
> >>> On Fri, 2007-02-23 at 06:55 -0500, Tres Seaver wrote:
> >>>> Roché Compaan wrote:
> >>>>> I'm curious, has anybody played around with the idea of caching ZCatalog
> >>>>> results and if I submitted a patch to do this would it be excepted?
> >>>>> I quickly coded some basic caching of results on a volatile attribute
> >>>>> and I was really surprised with the amount of cache hits I got
> >>>>> (especially with a Plone site that is a heavy user of the catalog)
> >>>> +1. I think using the 'ZCachable' stuff (e.g., adding a RAMCacheManager
> >>>> and associating a catalog to it) would be the sanest path here.
> >>> Cool idea. I haven't done any coding involving OFS.Cache though. Looking
> >>> at it briefly it looks like one can modify the catalog to subclass
> >>> OFS.Cacheable and then use the ZCacheable_get, ZCacheable_set and
> >>> ZCacheable_invalidate methods to interact with a cache manager. This
> >>> needs to be pretty explicit though. Are there any side effects that I
> >>> should guard against if the catalog subclasses OFS.Cache?
> > I don't think so. Here are some random thoughts on the idea:
> > - The 'searchResults' method must pass its keyword arguments as
> > part of the cache key.
> > - I don't know if there is a reasonable way to do 'mtime' for
> > the catalog: we would like to be able to get an mtime cheaply
> > for the BTrees (indexes, the 'data' container), but I don't know
> > if that is possible.
> > - The "right" place to do this feels like the 'searchResults' of
> > ZCatalog, just before it calls 'self._catalog.searchResults'.
> > - The CMF's catalog overrides 'searchResults', but calls it at
> > the end, so everything there should work.
In my prototype I also wired the caching into searchResults:
def searchResults(self, REQUEST=None, used=None, _merge=1, **kw):
cache_key = None
cache_key = self._makeCacheKey(args)
result = self._getCachedResult(cache_key)
return self._cacheResult(cache_key, self.search(args,
sort_index, reverse, sort_limit, _merge))
> Hmm, on further thought:
> - It isn't safe to stash persistent objects in the RAM Cache manager,
> because they can't be used safely from another database connection.
But the lazy map of brains isn't persistent?
> - The result set you get back from a query is a "lazy", which will
> be consumed by each client: no two clients will see the same
I don't follow. The Lazy will contain a set of document ids that will be
the same for all clients, not?
I got satisfactory results by storing results in a volatile attribute
(and they are not shared by clients).
I'm still curious to see what can be achieved with ZCacheable to extend
the lifetime of the cache.
Upfront Systems http://www.upfrontsystems.co.za
Zope-Dev maillist - Zope-Dev@zope.org
** No cross posts or HTML encoding! **
(Related lists -