One way to potentially soften the impact of cache busting by spiders might be to allow Zope to choose a particular ZODB connection based on request parameters (like sessionid or requesting ip address, or most likely user agent in the case of "legitimate" spiders). This is a modification to Zope that would be largely independent of ZODB, however, so might be more appropriate to discuss on zope-dev.
On Wed, 2005-06-08 at 10:20 +1000, Dylan Jay wrote: > I've been thinking about the problems of memory use. One use case that seems > hard is web crawlers. That will wake up a lot of objects for just a single > read possibly getting rid of many often used objects from the zodb cache. > Has anyone investigated the idea of a generational cache? > > Another idea I had was asymetric cache sizes. I believe that zope uses a > single thread if all requests are serial and just uses the other connections > when similtanious requests are handled. Since some threads will be less used > than others perhaps there could be cache settings so memory can be > minimised. > _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
