Thanks for your reply and sorry for the delayed response (been fighting fires).
On 01/24/2012 09:18 PM, Laurence Rowe wrote:
> On 24 January 2012 13:50, steve <st...@lonetwin.net> wrote:
>> Hi All,
>> Any pointers/comments would be appreciated.
> (Following up only on zodb-dev as I'm not subscribed to the other lists.)
> I'm guessing, but I suspect your load tests may only be reading from
> the ZODB so you rarely see any cache misses.
Yes, that's correct, we do test (mostly) read operations.
> The most important tuning paramaters for ZODB in respect to memory
> usage are the number of threads and the connection_cache_size. The
> connection_cache_size controls the number of persistent objects kept
> live in the interpreter at a time. It's a per-connection setting and
> as each thread needs its own connection. Memory usage increases
> proportionally to connection_cache_size * number of threads. Most
> people use either one or two threads per process with the ZODB. I know
> plone.recipe.zope2instance defaults to two threads per process, though
> I think this is only to avoid locking up in the case of Plone being
> configured to load an RSS feed from itself.
> The Python Global Interpreter Lock prevents threads from running
> concurrently, so with ZEO running so many threads per process is
> likely to be counter-productive. Try with one or two threads and
> perhaps up the connection_cache_size (though loading from the zeo
> cache is very quick you must ensure your working set fits in the
> connection cache or else you'll be loading the same objects every
> request). If your CPU usage goes down a lot it means your spending
> time waiting for objects to be loaded in the connection and you might
> want to run a couple more processes than you have cpus if you are
> running one thread per process.
Thanks for that information. That is helpful. Yes, I shall try and experiment
with the connection_cache_size.
> I also suggest you check that your mod_wsgi config is correctly
> specifying WSGIProcessGroup, see:
Yes, it is.
> To get more information about what is going on, try porting
> https://github.com/collective/collective.stats to work as WSGI
> middleware instead of hooking into Zope2's ZPublisher.
Hmm, this is interesting too. I'll try to see if I can port and fit it into our
Thanks for taking the time.
random spiel: http://lonetwin.net/
what i'm stumbling into: http://lonetwin.stumbleupon.com/
For more information about ZODB, see http://zodb.org/
ZODB-Dev mailing list - ZODB-Dev@zope.org