Re: [ZODB-Dev] Generational Caching
One way to potentially soften the impact of cache busting by spiders might be to allow Zope to choose a particular ZODB connection based on request parameters (like sessionid or requesting ip address, or most likely user agent in the case of legitimate spiders). This is a modification to Zope that would be largely independent of ZODB, however, so might be more appropriate to discuss on zope-dev. On Wed, 2005-06-08 at 10:20 +1000, Dylan Jay wrote: I've been thinking about the problems of memory use. One use case that seems hard is web crawlers. That will wake up a lot of objects for just a single read possibly getting rid of many often used objects from the zodb cache. Has anyone investigated the idea of a generational cache? Another idea I had was asymetric cache sizes. I believe that zope uses a single thread if all requests are serial and just uses the other connections when similtanious requests are handled. Since some threads will be less used than others perhaps there could be cache settings so memory can be minimised. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Generational Caching
[Chris McDonough] One way to potentially soften the impact of cache busting by spiders might be to allow Zope to choose a particular ZODB connection based on request parameters (like sessionid or requesting ip address, or most likely user agent in the case of legitimate spiders). This is a modification to Zope that would be largely independent of ZODB, however, so might be more appropriate to discuss on zope-dev. I don't know what generational caching means anyway, so that sounds like a fine idea to me too wink. When Jeremy and I were looking at alternative ZEO cache designs, several papers made the point that both recency and frequency of access should be taken into account. The ZODB memory cache is pure LRU now, considering only recency. Considering frequency of access too could/should be an effective way to avoid giving undue weight to one-shot requests. But that's a more complicated cache design than pure LRU, so who knows if it would be a _net_ win. Buying more RAM is an idea that just never gets old wink. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
