This is with whatever ZODB ships with Zope 2.8.5...
I have a Stepper (zopectl run on steroids) job that deals with lots of
After processing each one, Stepper does a transaction.get().commit(). I
thought this was enough to keep the object cache at a sane size, however
the job kept bombing out with MemoryErrors, and sure enough it was using
2 or 3 gigs of memory when that happened.
I fiddled about with the gc module and found that, sure enough, object
were being kept in memory. At a guess, I inserted something close to the
...after each 5,000 objects were processed (there are 60,000 objects in
Lo and behold, memory usage became sane.
Why is this step necessary? I thought transaction.get().commit() every
so often was enough to sort out the cache...
Simplistix - Content Management, Zope & Python Consulting
For more information about ZODB, see the ZODB Wiki:
ZODB-Dev mailing list - ZODB-Dev@zope.org