Short of using a memory debugger such as Valgrind or Purify, have you
considered looking for reference leaks? These may be the cause and
can be checked with pure python code. See how Lib/test/regrtest.py
handles the -R option.
n
--
On 1/24/07, Kristján V. Jónsson <[EMAIL PROTECTED]> wrote:
> Thanks, but the question is really, "how do I build a better debug hook than
> sys.getobjects?" so I argue this is a valid python-dev question.
>
> We have been using gc.get_objects() but it has several problems:
> 1) It returns all objects in the system. This results in a list so long that
> it often kills the system. Our system is of a scale that makes this very
> risky.
> 2) There is no way to frame certain operations and get just those objects
> that were created during their execution. In our case, we would like to get
> the server cluster running, then frame a certain operation to get a callback
> for all created objects, so that we could track that they were later
> destroyed correctly. I have done this previously by storing the id()s of all
> objects returned from gc.get_objects() and comparing them "before" and
> "after" but this suffers from 1) above, and the ambiguity of id() in the face
> of objects being created and destroyed.
>
>
> Working with the metaclasses sounds reasonable if one has the luxury of
> changing the entire codebase to use a different metaclass. It also doesn't
> work with old style classes (plenty of those), c extensions, and builtins.
>
> (I was a bit dismayed that I couldn't assign to object.__init__ post-hoc from
> a python script, I'm fairly sure that is possible in Ruby :) but I
> digress...)
>
> My latest attempt was to modify _PyObject_GC_TRACK(o) in objimpl.h, adding
> this final line:
> if (PyCCP_CreationHook) PyCCP_CreationHookFunc(o);\
>
> The function then looks like this:
>
> void PyCCP_CreationHookFunc(PyObject * obj)
> {
> if (PyCCP_CreationHook) {
> PyObject *result, *tmp = PyCCP_CreationHook;
> PyCCP_CreationHook = 0; //guard against recursion
> result = PyObject_CallFunctionObjArgs(PyCCP_CreationHook,
> obj, 0);
> Py_XDECREF(result);
> if (!result)
> PyErr_Clear();
> PyCCP_CreationHook = tmp;
> }
> }
>
> Doing this, and setting a no-op function as as the PyCCP_CreationHook, does
> seem to work for a while in the interactive python shell, but it then crashes
> and I haven't been able to work out the crash. In any case, doing stuff at
> the point of GC list insertion is very hairy, especially in the face of
> __del__ methods and such (of which we don't have many)
>
> I am waiting to get Rational Purify set up on my machine and then I'll maybe
> be able to understand the crash case better.
>
> Cheers,
> Kristján
>
>
> -----Original Message-----
> From: "Martin v. Löwis" [mailto:[EMAIL PROTECTED]
> Sent: 23. janúar 2007 23:32
> To: Kristján V. Jónsson
> Cc: 'python-dev@python.org'
> Subject: Re: [Python-Dev] Object creation hook
>
> Kristján V. Jónsson schrieb:
> > I am trying to insert a hook into python enabling a callback for all
> > just-created objects. The intention is to debug and find memory leaks,
> > e.g. by having the hook function insert the object into a WeakKeyDictionary.
>
> I'd like to point out that this isn't a python-dev question, but more
> appropriate for comp.lang.python (as it is of the "how do I x with
> Python?" kind).
>
> I would use a debug build, and use sys.getobjects to determine all
> objects and find leaks.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
