On 12/11/2012 05:33 PM, Jeroen Michiel wrote:
I'm having serious trouble getting my DB evolved to a new version. I'm
runnong a Grok 1.4 site using ZODB 3.10.2
The problem happens when I add a new index to a new catalog.
As soon as the index is added, a subscriber from zop.catalog (I believe)
will automatically loop over all objects in the DB trying to index them. For
some reason, it apparently tries to keep all these objects into memory,
while only a very small part of them are effectively need indexing, and even
then, indexing shouldn't touch them.
After some time of running I see the process taking 1.9GB of mem on windows,
(or 3G on linux), and then I first get these errors:
2012-12-11 16:52:56,617 ERROR [ZODB.Connection] Couldn't load state for
Traceback (most recent call last):
line 856, in setstate
line 910, in _setstate
line 612, in setGhostState
state = self.getState(pickle)
line 605, in getState
I have not a single clue why it would need that much memory.
I tried using savepoints, but that doesn't help.
How can I see what exactly is eating all that memory, where do I start
ANY help appreciated!
Well it loads too many objects in a single transaction.
Doing this after some iterations (10k?, depends on your object sizes)
if anyPersistentObject is not None:
#and clear picklecache
conn = anyPersistentObject._p_jar
Quote of the day:
A liberal is someone too poor to be a capitalist and too rich to be a
Zope3-users mailing list