On Wed, Dec 12, 2012 at 12:39:18AM -0800, Jeroen Michiel wrote: > Thanks for the reply! > > I already tried > transaction.savepoint() > every minute, but that didn't help: I only saw the memory usage dropping the > first time, but never after. > > I changed the code to what you suggested, but it still doesn't seem to help. > Something must be wrong somewhere along the line, but I don't have a clue > where to begin looking. > Would using something like guppy (or heapy, or what it's called) reveal > something?
(I tried to figure out guppy/heapy once, gave up.) > Could it be something about objects with circular references not being able > to be garbage-collected? > The objects in my DB are quite complex, so something like that might > actually be happening. Usually when a Python process eats too much memory, your server ends up in swappy death land. My gut feeling is that MemoryError means a corrupted pickle that tries to allocate a large amount of memory (e.g. multiple gigabytes) all in one go. Can you add a try:/except MemoryError: import pdb; pdb.set_trace() in there? See if the process memory usage is really big. Write down the object OID (ZODB.utils.u64(obj._p_oid)), try to load it from a separate Python script process (with the same sys.path, so custom classes are unplickleable): db = ZODB.DB.DB(ZODB.FileStorage.FileStorage('Data.fs', read_only=True)) conn = db.open() obj = conn.get(ZODB.utils.p64(0xXXXXX)) # creates a ghost try: obj._p_activate() # tries to load it except MemoryError: import pdb; pdb.set_trace() See if you get a memory error there. If so, try do disassemble the pickle (pickletools.dis) maybe -- if you've got pdb, you can find the pickle itself one stack frame up. Marius Gedminas -- I want patience, and I WANT IT NOW!
signature.asc
Description: Digital signature
_______________________________________________ Zope3-users mailing list Zope3-users@zope.org https://mail.zope.org/mailman/listinfo/zope3-users