Re: [Zope3-Users] MemoryError Evolving a ZODB

2012-12-18 Thread Jeroen Michiel
I found the actual issue! I am running a client/server setup: the client (C++ tool) runs tests on devices and pushes the results to the server (Grok) with REST requests. There was a mechanism that the server validated the results, and could reject them if they didn't validate, so the client would

Re: [Zope3-Users] MemoryError Evolving a ZODB

2012-12-14 Thread Jeroen Michiel
I was thinking along the same lines. I have found a spot in the code with circular references, and indeed (using heapy) it seems those are the objects (which happen to be quite big also) taking most of the memory. The main problem is now to get rid of them while staying within memory boundaries.

Re: [Zope3-Users] MemoryError Evolving a ZODB

2012-12-13 Thread Alexandre Garel
Le 12/12/2012 09:39, Jeroen Michiel a écrit : Thanks for the reply! I already tried transaction.savepoint() every minute, but that didn't help: I only saw the memory usage dropping the first time, but never after. I changed the code to what you suggested, but it still doesn't seem to help.

Re: [Zope3-Users] MemoryError Evolving a ZODB

2012-12-12 Thread Jeroen Michiel
Thanks for the reply! I already tried transaction.savepoint() every minute, but that didn't help: I only saw the memory usage dropping the first time, but never after. I changed the code to what you suggested, but it still doesn't seem to help. Something must be wrong somewhere along the line,

Re: [Zope3-Users] MemoryError Evolving a ZODB

2012-12-12 Thread Marius Gedminas
On Wed, Dec 12, 2012 at 12:39:18AM -0800, Jeroen Michiel wrote: Thanks for the reply! I already tried transaction.savepoint() every minute, but that didn't help: I only saw the memory usage dropping the first time, but never after. I changed the code to what you suggested, but it still

[Zope3-Users] MemoryError Evolving a ZODB

2012-12-11 Thread Jeroen Michiel
Hi, I'm having serious trouble getting my DB evolved to a new version. I'm runnong a Grok 1.4 site using ZODB 3.10.2 The problem happens when I add a new index to a new catalog. As soon as the index is added, a subscriber from zop.catalog (I believe) will automatically loop over all objects in