We have the need to mount dozens, if not hundreds of ZODBs in a single Zope instance.
With the current implementation, each ZODB connection has its own independent cache. This makes it very difficult to use the available memory efficiently for caching. As the caches are independent, a heavily used ZODB cannot use memory assigned to a lightly used ZODB. The LRU scheme would work much better when all connections mounted by the same root connection would share the cache with this root connection. I checked how I could implement this scheme: Mounted connections would not use their own cache but use a cache proxy instead. A cache proxy behaves identical to a true cache at the Python level. Internally, it forwards cache requests to a true cache prefixing oids with a distinguishing prefix. All cache proxies of mounted connections share the (true) cache of their root connection. Unfortunately, "cPersistence.c" currently makes assumptions about the cache implementation which now longer hold for cache proxies: that the cache object (as seen by the persistent object) maintains the number of non-ghosts and contains the "ring home". For a cache proxy, these items are not maintained by the proxy but by the associated true cache. I would like to use a more flexible way to couple the cache and the persistent object: via methods rather than fields. Instead of "ring_home" and "non_ghost_count", the cache interface would expose functions: void ghostify(cCache *, cPersistentObject *); void unghostify(cCache *, cPersistentObject *); void access(cCache *, cPersistentObject *); This would allow to handle cache proxies and true caches via the same interface. The added flexibility would also benefit future cache enhancements, such as: * choosing the cache implementation via configuration options * implementing enhanced cache replacement strategies * taking (approximate) size of objects into account For our current needs, I see an alternative solution which would not need to change the interface between cache and persistent object: the persistent object would see the true cache and not the proxy. The distinguishing prefix would then need to be stored in the persistent object itself. While sufficient for the current needs, it would not help with future caching enhancements. What is the best approach in your opinion? -- Dieter _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev