> That is after having run the 'preloading'. It seems that when the query
> takes forever, the non-ghost-size is slowly increasing (~100
> objects/second) while the 'size' stays the same. Once the query is done
> after having taken a few minutes, each subsequent run is instant and the
> ngsize doesn't grow. My naive question is: it has plenty of RAM, why does
> it not just load everything into the RAM?
It's actually not *that *slow - I didn't realize that everything seems to
stop while it's asking for cacheDetailSize. It seems to load about 10000
objects/minute, most of these being IFTreeSet/IFSet. This seems a bit
slow... if the index db has 750k objects in it, then it would take 75
minutes, at this rate, to read through it all, meaning an extensive query
would really take way too long...
Also my ZEO server is running locally, anyway, so the local socket transfer
speed shouldn't really be much slower than loading from the persistent
cache, should it? Either way it ends up loading from disk.
I don't quite understand why the zeoserver doesn't have any sort of
caching... hence my earlier thoughts of a memcachedb server to load all
this in RAM and to just run forever. Why would it not be a win in my
I'm pretty new to zodb so perhaps I don't understand a lot of the design
decisions very well and thus how best to take advantage of zodb, but I'm
willing to learn.
For more information about ZODB, see http://zodb.org/
ZODB-Dev mailing list - ZODB-Dev@zope.org