Hi All,

This is with whatever ZODB ships with Zope 2.8.5...

I have a Stepper (zopectl run on steroids) job that deals with lots of big objects.

After processing each one, Stepper does a transaction.get().commit(). I thought this was enough to keep the object cache at a sane size, however the job kept bombing out with MemoryErrors, and sure enough it was using 2 or 3 gigs of memory when that happened.

I fiddled about with the gc module and found that, sure enough, object were being kept in memory. At a guess, I inserted something close to the following:


...after each 5,000 objects were processed (there are 60,000 objects in total)

Lo and behold, memory usage became sane.

Why is this step necessary? I thought transaction.get().commit() every so often was enough to sort out the cache...



Simplistix - Content Management, Zope & Python Consulting
           - http://www.simplistix.co.uk
For more information about ZODB, see the ZODB Wiki:

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to