Re: [ZODB-Dev] Re: Memory Errors reading large ZODB
On Aug 2, 2007, at 9:33 AM, Jim Fulton wrote: On Aug 2, 2007, at 7:51 AM, Chris Withers wrote: Gary Poster wrote: you can call cache minimize after a threshold.. maybe every 100 iterations. sounds good, assuming you know you are not writing. I've used this trick loads, especially for huge datastructure migrations where writing is happening. I wonder why I haven't bumped into problems? I'm not sure what Gary was referring to. I don't think there are any problems with writing. OTOH, modified objects won't be removed from memory. Perhaps that was what he was referring to. Yup. Gary ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Memory Errors reading large ZODB
On Aug 2, 2007, at 7:51 AM, Chris Withers wrote: Gary Poster wrote: you can call cache minimize after a threshold.. maybe every 100 iterations. sounds good, assuming you know you are not writing. I've used this trick loads, especially for huge datastructure migrations where writing is happening. I wonder why I haven't bumped into problems? I'm not sure what Gary was referring to. I don't think there are any problems with writing. OTOH, modified objects won't be removed from memory. Perhaps that was what he was referring to. As Gary mentioned, you can use savepoints to cause modified objects to be saved to disk and thus removed from memory. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Memory Errors reading large ZODB
Gary Poster wrote: you can call cache minimize after a threshold.. maybe every 100 iterations. sounds good, assuming you know you are not writing. I've used this trick loads, especially for huge datastructure migrations where writing is happening. I wonder why I haven't bumped into problems? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Memory Errors reading large ZODB
On Aug 1, 2007, at 4:16 PM, Alan Runyan wrote: snip... line 96, in __setstate__ Persistent.__setstate__(self, state) MemoryError means you have run out of memory. check your jail or how much memory you have allocated. you can call cache minimize after a threshold.. maybe every 100 iterations. sounds good, assuming you know you are not writing. If you think you might be writing but don't intend to, and are willing to be sloppy, you probably could use transaction.abort() instead. If you were writing, the old trick was transaction.commit(1) and the new one is transaction.savepoint(). there are finer grain mechanisms to do this but i cant remember them off the top of my head. http://svn.zope.org/ZODB/trunk/src/ZODB/interfaces.py You are working with an old version of ZODB though, you said, so be careful looking at the trunk's interface-it might not describe precisely what you are using (savepoint is particularly pertinent here). Gary ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Memory Errors reading large ZODB
snip... > line 96, in > __setstate__ > Persistent.__setstate__(self, state) > MemoryError means you have run out of memory. check your jail or how much memory you have allocated. you can call cache minimize after a threshold.. maybe every 100 iterations. there are finer grain mechanisms to do this but i cant remember them off the top of my head. http://svn.zope.org/ZODB/trunk/src/ZODB/interfaces.py cheers alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev