One way to potentially soften the impact of cache busting by spiders
might be to allow Zope to choose a particular ZODB connection based on
request parameters (like sessionid or requesting ip address, or most
likely user agent in the case of "legitimate" spiders). This is a
modification to Zope that would be largely independent of ZODB, however,
so might be more appropriate to discuss on zope-dev.
On Wed, 2005-06-08 at 10:20 +1000, Dylan Jay wrote:
> I've been thinking about the problems of memory use. One use case that seems
> hard is web crawlers. That will wake up a lot of objects for just a single
> read possibly getting rid of many often used objects from the zodb cache.
> Has anyone investigated the idea of a generational cache?
> Another idea I had was asymetric cache sizes. I believe that zope uses a
> single thread if all requests are serial and just uses the other connections
> when similtanious requests are handled. Since some threads will be less used
> than others perhaps there could be cache settings so memory can be
For more information about ZODB, see the ZODB Wiki:
ZODB-Dev mailing list - ZODB-Dev@zope.org