Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

2010-12-23 Thread Bobby Impollonia
On Wed, Dec 22, 2010 at 9:26 PM, Victor Stinner vstin...@edenwall.com wrote: If the kernel doesn't do that for us, I suppose that the compiler or something else does it for us. GCC does this for you if you declare your function with __attribute__(signal). In general, the compiler doesn't know

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

2010-12-23 Thread Martin v. Löwis
Horrible or not, the existence of __attribute__(signal) seems to indicate that this is the case on some platform, or at least was historically. The signal attribute has an effect only on ATMEL AVR processors, according to the documentation (and according to my source grep). On other

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

2010-12-22 Thread Victor Stinner
Le lundi 20 décembre 2010 à 13:09 -0500, Alexander Belopolsky a écrit : What if the program crashes (again) in the signal handler before having a chance to restore registers? If the fault handler crashs, the registers are not restored. Also, can you point to documentation that describes

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-20 Thread Stefan Krah
Alexander Belopolsky alexander.belopol...@gmail.com wrote: On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl g.bra...@gmx.net wrote: .. In any case, this is coming pretty late; beta 2 is scheduled for this weekend, and even if this is something that only kicks in when all hope is lost anyway,

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-20 Thread Stefan Krah
Stefan Krah ste...@bytereef.org wrote: Another test is hanging indefinitely (Ubuntu 64-bit): $ ./python Lib/test/crashers/nasty_eq_vs_dict.py [hanging with no output] And this test does not report anything at all: $ ./python Lib/test/crashers/compiler_recursion.py [no output at all] $

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-20 Thread Victor Stinner
Le lundi 20 décembre 2010 08:22:40, vous avez écrit : Looking at your function list, my other concern is that you are calling Python API without holding the GIL, IIUC. In particular, you are accessing _PyThreadState_Current, which may not point to the current thread if the current thread

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER?environment variable

2010-12-20 Thread Stefan Krah
Victor Stinner vstin...@edenwall.com wrote: Le lundi 20 décembre 2010 12:08:35, Stefan Krah a écrit : I think the output is not particularly informative: $ ./python Lib/test/crashers/gc_inspection.py (object object at 0x7f01827ad870, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL,

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

2010-12-20 Thread Stefan Krah
Victor Stinner vstin...@edenwall.com wrote: Le lundi 20 décembre 2010 15:55:57, Stefan Krah a écrit : The backtrace is valid. Don't you think that this backtrace is more useful than just Segmentation fault? Perhaps I misunderstood, but I thought the purpose of the patch was to let

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

2010-12-20 Thread Alexander Belopolsky
On Mon, Dec 20, 2010 at 11:56 AM, Stefan Krah ste...@bytereef.org wrote: Victor Stinner vstin...@edenwall.com wrote: .. The fault handler helps developers because they don't have to have a Python compiled in debug mode and to run the application in a debugger (like gdb). If the developer is

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-20 Thread Martin (gzlist)
On 20/12/2010, Michael Foord fuzzy...@voidspace.org.uk wrote: On 19/12/2010 23:02, Victor Stinner wrote: I don't think that kill() is available on Windows. Can just use raise instead. But, is any of this change valid on Windows? http://msdn.microsoft.com/library/xdkz3x12 The SIGILL, SIGSEGV,

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

2010-12-20 Thread Victor Stinner
Le lundi 20 décembre 2010 18:22:48, Alexander Belopolsky a écrit : On Mon, Dec 20, 2010 at 11:56 AM, Stefan Krah ste...@bytereef.org wrote: Victor Stinner vstin...@edenwall.com wrote: .. The fault handler helps developers because they don't have to have a Python compiled in debug mode

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

2010-12-20 Thread Alexander Belopolsky
On Mon, Dec 20, 2010 at 12:51 PM, Victor Stinner vstin...@edenwall.com wrote: .. Are you sure that a signal handler changes the registers? At least Linux saves integer and float registers before calling the signal handler, and then restores them. What if the program crashes (again) in the

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER?environment variable

2010-12-20 Thread R. David Murray
On Mon, 20 Dec 2010 15:55:57 +0100, Stefan Krah ste...@bytereef.org wrote: Victor Stinner vstin...@edenwall.com wrote: Stefan Krah wrote: I think the output is not particularly informative: $ ./python Lib/test/crashers/gc_inspection.py (object object at 0x7f01827ad870, NULL, NULL,

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER?environment variable

2010-12-20 Thread Stefan Krah
Victor Stinner vstin...@edenwall.com wrote: Le lundi 20 décembre 2010 12:08:35, Stefan Krah a écrit : Another test is hanging indefinitely (Ubuntu 64-bit): $ ./python Lib/test/crashers/nasty_eq_vs_dict.py [hanging with no output] Oh, I understood. I always test with Python compiled

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER?environment variable

2010-12-20 Thread Martin v. Löwis
Do you have an example bug where this patch helps in finding the precise location of a segfault? How is 'line 29 in g' not more precise than 'Segmentation fault'? I think Stefan is treating precise location as an absolute property, not a relative one. The precise location would be the very

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-20 Thread Martin v. Löwis
The GIL is likely held by a different thread, then. _PyThreadState_Current will point to the state of this other thread. I tested when the GIL released: the fault handler is unable to retrieve the thread state and so the backtrace is not printed. Which thread state should be retrieve? I

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Martin v. Löwis
This is also what I think. Installing a signal handler is a fairly drastic action, and I don't think the code has been sufficiently reviewed yet. How much more review should it receive? There should be at least one reviewer with an established track record of being interested/knowledgable

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Antoine Pitrou
Le dimanche 19 décembre 2010 à 19:53 +0100, Martin v. Löwis a écrit : This is also what I think. Installing a signal handler is a fairly drastic action, and I don't think the code has been sufficiently reviewed yet. How much more review should it receive? There should be at least

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Alexander Belopolsky
On Sun, Dec 19, 2010 at 2:20 PM, Antoine Pitrou solip...@pitrou.net wrote: .. There should be at least one reviewer with an established track record of being interested/knowledgable in the POSIX APIs and cross-platform aspects of it. As I said, any of the Twisted guys would qualify. James

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Victor Stinner
Le dimanche 19 décembre 2010 à 19:53 +0100, Martin v. Löwis a écrit : There should be at least one reviewer with an established track record of being interested/knowledgable in the POSIX APIs and cross-platform aspects of it. Functions used by the fault handler: - write() - PyUnicode_Check()

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Victor Stinner
Le samedi 18 décembre 2010 à 17:23 +0100, Georg Brandl a écrit : Well, without a closer I assume that for some crashes it's just not possible anymore for the Python interpreter to even print out the traceback? The worst case occurs if the frame objects are corrupted, eg. if the filename of a

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Victor Stinner
Le samedi 18 décembre 2010 à 13:55 -0500, Alexander Belopolsky a écrit : I am -1 on the feature as written. I would be -0 if it did not install signal handlers by default and even better was implemented in a separate module, not in core. I think that the feature is useless if it is disabled

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Scott Dial
On 12/18/2010 8:50 AM, James Y Knight wrote: I think instead of calling abort() to kill the process, you should: - install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is not present, explicitly reset the action to SIG_DFL and unblock first thing upon entering the handler),

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Victor Stinner
Le dimanche 19 décembre 2010 à 20:20 +0100, Antoine Pitrou a écrit : Can you clarify why you think those signal handlers fall into that description? After all, SIGINT, SIGILL and friends only get triggered in catastrophic conditions. SIGSEGV, not SIGINT Victor

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Antoine Pitrou
On Sun, 19 Dec 2010 23:37:58 +0100 Victor Stinner victor.stin...@haypocalc.com wrote: Le dimanche 19 décembre 2010 à 20:20 +0100, Antoine Pitrou a écrit : Can you clarify why you think those signal handlers fall into that description? After all, SIGINT, SIGILL and friends only get triggered

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Victor Stinner
Le samedi 18 décembre 2010 à 08:50 -0500, James Y Knight a écrit : I think instead of calling abort() to kill the process, you should: - install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is not present, explicitly reset the action to SIG_DFL and unblock first thing

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Victor Stinner
Le lundi 20 décembre 2010 à 00:02 +0100, Victor Stinner a écrit : Another possible approach is to restore the previous handler in the fault handler, and don't call abort(). The fault should occur again, and so the previous signal handler will be called. I implemented this simple approach in

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Martin v. Löwis
Functions used by the fault handler: - write() - PyUnicode_Check() - PyFrame_GetLineNumber() - DebugBreak() (Windows, in debug mode, only) - abort() - (macro) PyUnicode_GET_SIZE() and PyUnicode_AS_UNICODE() - PyUnicode_Check(), PyFrame_Check() - PyFrame_GetLineNumber() -

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Martin v. Löwis
What? Is it a myth or does Python really support multiple interpreters in the same process? How is it possible? Who uses this? [Not sure if you are joking] There is certainly some support for multiple interpreters in Python; the most prominent user of this feature is mod_python. Regards,

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Michael Foord
On 19/12/2010 23:02, Victor Stinner wrote: Le samedi 18 décembre 2010 à 08:50 -0500, James Y Knight a écrit : I think instead of calling abort() to kill the process, you should: - install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is not present, explicitly reset the

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Victor Stinner
Le lundi 20 décembre 2010 à 02:05 +0100, Martin v. Löwis a écrit : The problem is that merely being POSIX compliant would not be enough to allow calling a function in a signal handler. Instead, the function *also* needs to be async-signal safe. Yes, this issue was fixed in an older version of

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-19 Thread Martin v. Löwis
Looking at your function list, my other concern is that you are calling Python API without holding the GIL, IIUC. In particular, you are accessing _PyThreadState_Current, which may not point to the current thread if the current thread has released the GIL. Ah? Where does

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-18 Thread Georg Brandl
Am 18.12.2010 01:55, schrieb Victor Stinner: Hi, In 2008, I proposed a patch to raise a Python exception on SIGSEVG signal. In some cases, it's possible to catch the exception, log it, and continue the execution. But because in some cases, the Python internal state is corrupted, the idea

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-18 Thread James Y Knight
On Dec 17, 2010, at 7:55 PM, Victor Stinner wrote: Hi, In 2008, I proposed a patch to raise a Python exception on SIGSEVG signal. In some cases, it's possible to catch the exception, log it, and continue the execution. But because in some cases, the Python internal state is corrupted,

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-18 Thread Victor Stinner
Le 18/12/2010 13:21, Georg Brandl a écrit : I very much like having a traceback on (some) segmentation faults, Why do you say some segmentation faults? but it's clear there needs to be a way to turn it off. An environment variable seems to be the obvious choice (for the reasons you stated for

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-18 Thread Georg Brandl
Am 18.12.2010 14:57, schrieb Victor Stinner: Le 18/12/2010 13:21, Georg Brandl a écrit : I very much like having a traceback on (some) segmentation faults, Why do you say some segmentation faults? Well, without a closer I assume that for some crashes it's just not possible anymore for the

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-18 Thread Georg Brandl
Am 18.12.2010 14:57, schrieb Victor Stinner: Le 18/12/2010 13:21, Georg Brandl a écrit : I very much like having a traceback on (some) segmentation faults, Why do you say some segmentation faults? but it's clear there needs to be a way to turn it off. An environment variable seems to be

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-18 Thread Antoine Pitrou
On Sat, 18 Dec 2010 17:50:38 +0100 Georg Brandl g.bra...@gmx.net wrote: Am 18.12.2010 14:57, schrieb Victor Stinner: Le 18/12/2010 13:21, Georg Brandl a écrit : I very much like having a traceback on (some) segmentation faults, Why do you say some segmentation faults? but it's clear there

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-18 Thread Alexander Belopolsky
On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl g.bra...@gmx.net wrote: .. In any case, this is coming pretty late; beta 2 is scheduled for this weekend, and even if this is something that only kicks in when all hope is lost anyway, it is a new feature.  I should like to hear approval from a

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-18 Thread Martin v. Löwis
Am 18.12.2010 19:55, schrieb Alexander Belopolsky: On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl g.bra...@gmx.net wrote: .. In any case, this is coming pretty late; beta 2 is scheduled for this weekend, and even if this is something that only kicks in when all hope is lost anyway, it is a

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

2010-12-18 Thread Antoine Pitrou
On Sat, 18 Dec 2010 20:36:50 +0100 Martin v. Löwis mar...@v.loewis.de wrote: Am 18.12.2010 19:55, schrieb Alexander Belopolsky: On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl g.bra...@gmx.net wrote: .. In any case, this is coming pretty late; beta 2 is scheduled for this weekend, and even