On Wed, Apr 01, 2009 at 11:46:01AM +0200, Santi Camps wrote: > I'm having memory problems and, after some debugging with gc, I see > some estrange behaviours with BTreeFolder2 (or with BTrees itself, I'm > not sure). I've a BTreeFolder2 with 40000 objects, more o less. > Some of them are emails. Just accessing the container, about 700 > mails are loaded in memory !! > > from AccessControl.SecurityManagement import newSecurityManager > from Testing.makerequest import makerequest > user = app.beta.acl_users.getUser('manager').__of__(app.beta.acl_users) > newSecurityManager({}, user) > app = makerequest(app) > km = app.beta > km.portal_repository._tree > import gc > objects = gc.get_objects() > objects2 = [obj for obj in objects if getattr(obj, '__class__', None)]
(I'm a bit surprised that you get objects without a __class__ attribute -- could you elaborate about those? Old-style class instances? Extension types?) > emails = [obj for obj in objects2 if 'mail' in obj.__class__.__name__.lower()] How many different classes are there with 'mail' in the (lowercased) class name? Do they all inherit from Persistent? > print len(emails) > 697 > > Following gc references, I see this mails are referenced from a > OOBucket, that is referenced by another OOBucket, ... I assume those mails are values rather than keys. > and some of them > are referenced from persistent.PickleCache (instead I've configured > cache-size = 0 in zope.conf). Seems that some buckets and its > referenced objects are readed during btree loading. So, the garbage > collector never clears this from memory, and after some hours of > intensive work, the server RAM is finished > > Is that normal ? I'm making some mistake ? Anybody knows any way to solve > it ? Your description would make perfect sense to me if your mail classes weren't subclasses of Persistent (and the fix would be: make them subclass Persistent). Marius Gedminas -- I used to (somewhat) sneer at people who describe themselves thus: "I program HTML". Then I tried to make a web-site look as I wanted it to (ie. not grotesquely ugly), wow - perhaps they were onto something. Some think there is a pile of broken mess in our desktop, but at least you know where you are with a GtkHBox. -- Michael Meeks
signature.asc
Description: Digital signature
_______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev