On Fri, 2007-02-23 at 12:09 -0500, Tres Seaver wrote:
> Hash: SHA1
> Tres Seaver wrote:
> > Roché Compaan wrote:
> >>> On Fri, 2007-02-23 at 06:55 -0500, Tres Seaver wrote:
> >>>> Roché Compaan wrote:
> >>>>> I'm curious, has anybody played around with the idea of caching ZCatalog
> >>>>> results and if I submitted a patch to do this would it be excepted?
> >>>>>
> >>>>> I quickly coded some basic caching of results on a volatile attribute
> >>>>> and I was really surprised with the amount of cache hits I got
> >>>>> (especially with a Plone site that is a heavy user of the catalog)
> >>>> +1.  I think using the 'ZCachable' stuff (e.g., adding a RAMCacheManager
> >>>> and associating a catalog to it) would be the sanest path here.
> >>> Cool idea. I haven't done any coding involving OFS.Cache though. Looking
> >>> at it briefly it looks like one can modify the catalog to subclass
> >>> OFS.Cacheable and then use the ZCacheable_get, ZCacheable_set and
> >>> ZCacheable_invalidate methods to interact with a cache manager. This
> >>> needs to be pretty explicit though. Are there any side effects that I
> >>> should guard against if the catalog subclasses OFS.Cache?
> > 
> > I don't think so.  Here are some random thoughts on the idea:
> > 
> >  -  The 'searchResults' method must pass its keyword arguments as
> >     part of the cache key.
> > 
> >  - I don't know if there is a reasonable way to do 'mtime' for
> >    the catalog:  we would like to be able to get an mtime cheaply
> >    for the BTrees (indexes, the 'data' container), but I don't know
> >    if that is possible.
> > 
> >  - The "right" place to do this feels like the 'searchResults' of
> >    ZCatalog, just before it calls 'self._catalog.searchResults'.
> > 
> >  - The CMF's catalog overrides 'searchResults', but calls it at
> >    the end, so everything there should work.

In my prototype I also wired the caching into searchResults:

    def searchResults(self, REQUEST=None, used=None, _merge=1, **kw):
        cache_key = None
        if args:
            cache_key = self._makeCacheKey(args)

        result = self._getCachedResult(cache_key)
        if result:
            return result

        return self._cacheResult(cache_key, self.search(args,
sort_index, reverse, sort_limit, _merge))

> Hmm, on further thought:
>  - It isn't safe to stash persistent objects in the RAM Cache manager,
>    because they can't be used safely from another database connection.

But the lazy map of brains isn't persistent?

>  - The result set you get back from a query is a "lazy", which will
>    be consumed by each client:  no two clients will see the same
>    thing.

I don't follow. The Lazy will contain a set of document ids that will be
the same for all clients, not?

I got satisfactory results by storing results in a volatile attribute
(and they are not shared by clients).

I'm still curious to see what can be achieved with ZCacheable to extend
the lifetime of the cache.

Roché Compaan
Upfront Systems                   http://www.upfrontsystems.co.za

Zope-Dev maillist  -  Zope-Dev@zope.org
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope )

Reply via email to