Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-18 Thread Nick Coghlan
Greg Ewing wrote: Martin v. Löwis wrote: when some caller of PyEval_EvalFrameEx still carries a pointer to some object that got deleted, and then still some code can get hold of the then-deleted object. I seem to have missed the beginning of this discussion. I don't see what the problem is

Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-18 Thread Neil Schemenauer
Nick Coghlan [EMAIL PROTECTED] wrote: The problem is calls to Py_DECREF(self-attr) where some of the code invoked by __del__ manages to find a way back around to reference self-attr and gets access to a half-deleted object. Don't you mean __del__ manages to find a way back around to self? If

Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-18 Thread Amaury Forgeot d'Arc
Hello, Neil Schemenauer wrote: Nick Coghlan [EMAIL PROTECTED] wrote: The problem is calls to Py_DECREF(self-attr) where some of the code invoked by __del__ manages to find a way back around to reference self-attr and gets access to a half-deleted object. Don't you mean __del__ manages to

Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-18 Thread Jeffrey Yasskin
On Feb 17, 2008 12:29 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: Jeffrey Yasskin wrote: On Feb 16, 2008 3:12 PM, Amaury Forgeot d'Arc [EMAIL PROTECTED] wrote: Should we however intensively search and correct all of them? Is there a clever way to prevent these problems globally, for

Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-18 Thread Neil Schemenauer
On Mon, Feb 18, 2008 at 05:48:57PM +0100, Amaury Forgeot d'Arc wrote: For example, in exception.c, BaseException_init() starts with the instruction: Py_DECREF(self-args); this may call __del__ on self-args Ah, I understand now. We are not talking about tp_dealloc methods (the GC takes

[Python-Dev] Py_CLEAR to avoid crashes

2008-02-18 Thread Jim Jewett
A simple way to do this would be to push objects whose refcounts had reached 0 onto a list instead of finalizing them immediately, and have PyEval_EvalFrameEx periodically swap in a new to-delete list and delete the objects on the old one. Some of the memory management threads discussed

Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-18 Thread Martin v. Löwis
That sucks. Most Py_DECREF calls are probably okay but it's going to be hard to find the ones that are not. Methinks that egrep 'DECREF\([a-zA-Z0-9_]-[a-zA-Z0-9_]+\)' */*.c gives a good overview - you should never DECREF a variable on heap. In the trunk, this command finds 36 candidates.

Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-18 Thread Neil Schemenauer
I wrote: Most Py_DECREF calls are probably okay but it's going to be hard to find the ones that are not. I suppose Py_DECREF is not the only source of trouble. Many calls to the Python API can end up calling arbitrary user code (via __getattr__, __getitem__, etc.). Whenever an object does

Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-17 Thread Jeffrey Yasskin
On Feb 16, 2008 3:12 PM, Amaury Forgeot d'Arc [EMAIL PROTECTED] wrote: Should we however intensively search and correct all of them? Is there a clever way to prevent these problems globally, for example by delaying finalizers just a little? A simple way to do this would be to push objects

Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-17 Thread Martin v. Löwis
Jeffrey Yasskin wrote: On Feb 16, 2008 3:12 PM, Amaury Forgeot d'Arc [EMAIL PROTECTED] wrote: Should we however intensively search and correct all of them? Is there a clever way to prevent these problems globally, for example by delaying finalizers just a little? A simple way to do this

[Python-Dev] Py_CLEAR to avoid crashes

2008-02-16 Thread Amaury Forgeot d'Arc
Hello, I think I found a prolific vein of potential crashes throughout the python interpreter. The idea is inspired from the discussion Issue1020188 and is quite simple: find a Py_DECREF(xxx-yyy), where xxx represents any kind of Python object, and craft a __del__ method so that the same function

Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-16 Thread Brett Cannon
On Feb 16, 2008 3:12 PM, Amaury Forgeot d'Arc [EMAIL PROTECTED] wrote: Hello, I think I found a prolific vein of potential crashes throughout the python interpreter. The idea is inspired from the discussion Issue1020188 and is quite simple: find a Py_DECREF(xxx-yyy), where xxx represents any

Re: [Python-Dev] Py_CLEAR to avoid crashes

2008-02-16 Thread Nick Coghlan
Brett Cannon wrote: On Feb 16, 2008 3:12 PM, Amaury Forgeot d'Arc [EMAIL PROTECTED] wrote: Is there a clever way to prevent these problems globally, for example by delaying finalizers just a little? Beats me. Finalizers can be delayed 'just a little' with a macro called Py_CLEAR ;) (in