Re: [Zope] Zope/Cache/Memory
kevin7kal wrote at 2006-12-6 14:30 -0500: I have a database that is just under 200K objects with plans that it will grow larger. The objects are created by using another object with methods that read files and create the objects from the data in the files. When creating the objects, I find I have to manually clear the cache or else zope eventually crashes with an out of memory error. There are 4GB of ram on the server. The same issue arises when I catalog the objects. I am trying maintain as much speed as possible, so I do not want to catalog until the end of the import. Is there a way to programmatically clear the cache during a large indexing to avoid out of memory errors? A ZODB Connection has the method cacheGC() which you can call to start a cache garbage collection. As others already pointed out, it is implicitely called during large catalog operations -- controlled by the so called subtransaction threshold. Note that in early versions of ZODB 3.4, there was a bug preventing the call to cacheGC at subtransaction boundaries. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope/Cache/Memory
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On 6. Dezember 2006 14:30:40 -0500 kevin7kal [EMAIL PROTECTED] wrote: I have a database that is just under 200K objects with plans that it will Is there a way to programmatically clear the cache during a large indexing to avoid out of memory errors? You're indexing with the ZCatalog? Are you sure that you're using subtransactions (or savepoints as they are called now). There is a ZCatalog parameter that controls this behavior. For large indexing operations this might cause the OutOfMember error. When using subtransactions changes will be written to disk after N objects. Please check that. - -aj -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iD8DBQFFdykGCJIWIbr9KYwRAsz/AJ9kUeK+Vd8guq8EnxR+bajzKvAPGgCeNG1L jxZ9z8CZUjYbGAsTxZmWi+A= =YIf2 -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope/Cache/Memory
- Original Message - From: kevin7kal [EMAIL PROTECTED] To: zope@zope.org Sent: Wednesday, December 06, 2006 2:30 PM Subject: [Zope] Zope/Cache/Memory I have a database that is just under 200K objects with plans that it will grow larger. The objects are created by using another object with methods that read files and create the objects from the data in the files. When creating the objects, I find I have to manually clear the cache or else zope eventually crashes with an out of memory error. There are 4GB of ram on the server. The same issue arises when I catalog the objects. I am trying maintain as much speed as possible, so I do not want to catalog until the end of the import. Make sure you have correctly set the cache parameters in zope.conf, and make sure that the ZCatalog subtransactions are enabled - and you can play around with the threshold values to improve performance (I am assuming that the database you mentioned was a ZCatalog). hth Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
