Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

[Bobby Impollonia]

On Wed, Dec 22, 2010 at 9:26 PM, Victor Stinner vstin...@edenwall.com wrote:
 If the kernel doesn't do that for us, I
 suppose that the compiler or something else does it for us.

GCC does this for you if you declare your function with
__attribute__(signal). In general, the compiler doesn't know that a
function will be used as a signal and no portable way to indicate
this.

 Anyway, it would just be horrible if a signal handler was responsible to
 save/restore integer registers.

Horrible or not, the existence of __attribute__(signal) seems to
indicate that this is the case on some platform, or at least was
historically. I don't know whether it's the case now on any platform
that Python supports.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

[Martin v. Löwis]

 Horrible or not, the existence of __attribute__(signal) seems to
 indicate that this is the case on some platform, or at least was
 historically.

The signal attribute has an effect only on ATMEL AVR processors,
according to the documentation (and according to my source grep).
On other processors, gcc typically implements
__attribute__((interrupt)), meaning that the function is an interrupt
handler - something that you cannot express in regular C. This may
involve re-enabling interrupts on entry, but may also involve other
code generation aspects (such as using iret instead of ret on x86).

For AVR, there was apparently some compatibility issue, introducing
the need for two different ways to declare an ISR:
__attribute__((interrupt)) causes interrupts to be re-enabled on
entry (using the sei instruction); __attribute__((signal)) keeps
them disabled.

So this has nothing to do with Unix signals.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

[Victor Stinner]

Le lundi 20 décembre 2010 à 13:09 -0500, Alexander Belopolsky a écrit :
 What if the program crashes (again) in the signal handler before
 having a chance to restore registers?

If the fault handler crashs, the registers are not restored.

 Also, can you point to documentation that describes register restoring 
 behavior?

I found an article telling that the kernel saves integer and float
registers when calling a signal handler, but I am unable to find it
again, sorry. And I cannot find an article telling the opposite.

Anyway, it would just be horrible if a signal handler was responsible to
save/restore integer registers. If the kernel doesn't do that for us, I
suppose that the compiler or something else does it for us.

Eg. signal_default_int_handler(), the Python handler of SIGINT signal,
doesn't save/restore registers, even if it erases many of them.

 Because I had actual experience with that using R.  Your code may be
 better, but R brings up a dialog on segfault asking ...

You cannot compare a simple handler using simple signal-safe functions
like write(), and a complex handler calling graphical functions. It is
not safe to call graphical functions in a signal handler.

 (I actually use RPy which loads R into  Python as a shared library, 
 so the two handlers will conflict anyways.)

It depends if R handler calls the previous handler. My fault handler is
chained with the previous handler (it calls indirectly the previous
handler). If R fault handler does the same, all handlers will be
executed correctly (I don't think that the execution order matter).

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Stefan Krah]

Alexander Belopolsky alexander.belopol...@gmail.com wrote:
 On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl g.bra...@gmx.net wrote:
 ..
  In any case, this is coming pretty late; beta 2 is scheduled for this
  weekend, and even if this is something that only kicks in when all hope
  is lost anyway, it is a new feature.  I should like to hear approval
  from a few more devs before I will let this go into 3.2.
 
 
 I am -1 on the feature as written.  I would be -0 if it did not
 install signal handlers by default and even better was implemented in
 a separate module, not in core.


I would not want this to be the default either. I think the output
is not particularly informative:

$ ./python Lib/test/crashers/gc_inspection.py 
(object object at 0x7f01827ad870, NULL, NULL, NULL, NULL, NULL, 
NULL, NULL, NULL, NULL)
Fatal Python error: Segmentation fault

Traceback (most recent call first):
  File Lib/test/crashers/gc_inspection.py, line 29 in g
  File Lib/test/crashers/gc_inspection.py, line 32 in module
Segmentation fault
Segmentation fault


Another test is hanging indefinitely (Ubuntu 64-bit):

$ ./python Lib/test/crashers/nasty_eq_vs_dict.py
[hanging with no output]



It's hard to get signal handlers right across multiple platforms; even
SIGINT catching does not work properly on OpenBSD:

http://bugs.python.org/issue8714


In short, I agree that having more signal handlers by default is not a
good idea.



Stefan Krah



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Stefan Krah]

Stefan Krah ste...@bytereef.org wrote:
 Another test is hanging indefinitely (Ubuntu 64-bit):
 
 $ ./python Lib/test/crashers/nasty_eq_vs_dict.py
 [hanging with no output]


And this test does not report anything at all:

$ ./python Lib/test/crashers/compiler_recursion.py 
[no output at all]
$ echo $?
0


Without the patch:

$ ./python Lib/test/crashers/compiler_recursion.py 
Segmentation fault
$ echo $?
139



Stefan Krah


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Victor Stinner]

Le lundi 20 décembre 2010 08:22:40, vous avez écrit :
  Looking at your function list, my other concern is that you are calling
  Python API without holding the GIL, IIUC. In particular, you are
  accessing _PyThreadState_Current, which may not point to the current
  thread if the current thread has released the GIL.
  
  Ah? Where does _PyThreadState_Current point to if the GIL is not hold
  when the fault handler is called?
 
 The GIL is likely held by a different thread, then.
 _PyThreadState_Current will point to the state of this other thread.

I tested when the GIL released: the fault handler is unable to retrieve the 
thread state and so the backtrace is not printed. Which thread state should be 
retrieve? I don't know yet: maybe the last active thread state?

It doesn't look trivial to fix this issue because if there is no thread (only 
the main thread), release the GIL clears the thread state pointer (to it to 
NULL) without storing it somewhere else (whereas with multiple threads, the 
last active thread is stored in gil_last_holder. I should ask Antoine (or 
someone one) how the new GIL works.

 Of this thread? I don't think so. gil_last_holder might also refer to
 a different thread.

Another variable might be created to store the (pointer to the) last active 
thread state.

  I don't think that it will possible the acquire the GIL in
  Py_FatalError() or in the fault handler.
 
 I agree.

We all love deadlocks :-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER?environment variable

[Stefan Krah]

Victor Stinner vstin...@edenwall.com wrote:
 Le lundi 20 décembre 2010 12:08:35, Stefan Krah a écrit :
  I think the output is not particularly informative:
  
  $ ./python Lib/test/crashers/gc_inspection.py
  (object object at 0x7f01827ad870, NULL, NULL, NULL, NULL, NULL,
  NULL, NULL, NULL, NULL) Fatal Python error: Segmentation fault
  
  Traceback (most recent call first):
File Lib/test/crashers/gc_inspection.py, line 29 in g
File Lib/test/crashers/gc_inspection.py, line 32 in module
  Segmentation fault
  Segmentation fault
 
 The backtrace is valid. Don't you think that this backtrace is more useful 
 than just Segmentation fault?

Perhaps I misunderstood, but I thought the purpose of the patch was to
let developers act more quickly on bug reports. I wonder if this output
really speeds up the process.

Do you have an example bug where this patch helps in finding the precise
location of a segfault?


 Patch version 9 doesn't call the previous handler. Please retry with patch 
 version 10 (which call the previous handler).

I did use version 10. I've verified the same behavior with a fresh py3k
checkout and this patch:

$ md5sum segfault_handler-10.patch
e1b5a9439d2b51d0a63f701dd694796d  segfault_handler-10.patch


My machine currently has a load average of 2. Perhaps you'll be able to
reproduce it if you crank up the load average.



Stefan Krah


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

[Stefan Krah]

Victor Stinner vstin...@edenwall.com wrote:
 Le lundi 20 décembre 2010 15:55:57, Stefan Krah a écrit :
   The backtrace is valid. Don't you think that this backtrace is more
   useful than just Segmentation fault?
  
  Perhaps I misunderstood, but I thought the purpose of the patch was to
  let developers act more quickly on bug reports. I wonder if this output
  really speeds up the process.
 
 The fault handler helps developers because they don't have to have a Python 
 compiled in debug mode and to run the application in a debugger (like gdb).
 
 If the developer is unable to reproduce an error, because it's an Heisenbug, 
 or because the developer doesn't have the same OS, libraries and/or 
 applications, the fault handler helps the developer to isolate the problem.

This is what I'm sceptical about. I think the tracebacks are don't carry
enough information for that.


 Any bug report which only contain Segmentation fault information. But I 
 cannot give you a revelant example because users are too shy to report an 
 issue if they don't have more information than just the program crashed. 
 With a fault handler displaying the Python backtrace, I hope that more 
 *users* 
 will report bugs.

I wonder. It would be fine to make the tracebacks optional and tell the
user to rerun the program with the proper environment variable set.


  I did use version 10. I've verified the same behavior with a fresh py3k
  checkout and this patch: (...)
  
  My machine currently has a load average of 2. Perhaps you'll be able to
  reproduce it if you crank up the load average.
 
 What is your OS (name, version)? I only tested crashers on Linux (Debian Sid).

Ubuntu 8.10, 64-bit, real hardware.


I also tested on FreeBSD-8.0-qemu:

$ ./python Lib/test/crashers/compiler_recursion.py 
Illegal instruction: 4 (core dumped)


$ ./python Lib/test/crashers/nasty_eq_vs_dict.py 
[hanging with no output]



Stefan Krah


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

[Alexander Belopolsky]

On Mon, Dec 20, 2010 at 11:56 AM, Stefan Krah ste...@bytereef.org wrote:
 Victor Stinner vstin...@edenwall.com wrote:
..
 The fault handler helps developers because they don't have to have a Python
 compiled in debug mode and to run the application in a debugger (like gdb).

 If the developer is unable to reproduce an error, because it's an Heisenbug,
 or because the developer doesn't have the same OS, libraries and/or
 applications, the fault handler helps the developer to isolate the problem.

 This is what I'm sceptical about. I think the tracebacks are don't carry
 enough information for that.


My concern is that the traceback generated by a signal handler may
actually contain *less* information that what would otherwise be
available from the core dump or other system diagnostic facilities.
Whenever you continue to execute code after  a fault occurs (even in
an alternative stack), you change register values and run the risk
that the program will not be able to produce a core dump at all or
will produce a core dump with misleading information in it.

I think Stefan's approach is the correct one: before we agree on this
feature, the proponents should present actual crashers for which
reports generated by the patch are actually helpful.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Martin (gzlist)]

On 20/12/2010, Michael Foord fuzzy...@voidspace.org.uk wrote:
 On 19/12/2010 23:02, Victor Stinner wrote:
 I don't think that kill() is available on Windows.

Can just use raise instead.

But, is any of this change valid on Windows?

http://msdn.microsoft.com/library/xdkz3x12
The SIGILL, SIGSEGV, and SIGTERM signals are not generated under
Windows NT. They are included for ANSI compatibility.

And the last signal this concerns is explicitly skipped in the tests
on the latest patch under windows with the note SIGFPE cannot be
catched on Windows. Signals are a posix compatibility hack on win32,
they're unlikely to do what you want.

 Thanks to Brian Curtin there is now an os.kill() on Windows, but it
 doesn't handle the full range of signals.

Which uses GenerateConsoleCtrlEvent and is completely the wrong thing.

Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

[Victor Stinner]

Le lundi 20 décembre 2010 18:22:48, Alexander Belopolsky a écrit :
 On Mon, Dec 20, 2010 at 11:56 AM, Stefan Krah ste...@bytereef.org wrote:
  Victor Stinner vstin...@edenwall.com wrote:
 ..
 
  The fault handler helps developers because they don't have to have a
  Python compiled in debug mode and to run the application in a debugger
  (like gdb).
  
  If the developer is unable to reproduce an error, because it's an
  Heisenbug, or because the developer doesn't have the same OS, libraries
  and/or applications, the fault handler helps the developer to isolate
  the problem.
  
  This is what I'm sceptical about. I think the tracebacks are don't carry
  enough information for that.

Yes, in some cases the backtrace doesn't help. But why don't you want to 
display the backtrace if it can help in some other cases?

 My concern is that the traceback generated by a signal handler may
 actually contain *less* information that what would otherwise be
 available from the core dump or other system diagnostic facilities.

I changed my patch to call the previous signal handler starting at version 10. 
So the core dump function or OS fault handler will be called two. Said 
differently, it only gives *more* information, not less.

If you have a better OS fault handler than Python fault handler, you should 
disable Python fault handler by setting the environment variable.

 Whenever you continue to execute code after  a fault occurs (even in
 an alternative stack), you change register values and run the risk
 that the program will not be able to produce a core dump at all or
 will produce a core dump with misleading information in it.

Are you sure that a signal handler changes the registers? At least Linux saves 
integer and float registers before calling the signal handler, and then 
restores them.

The fault handler only changes some registers and write some bytes on the 
stack. The stack pointer is restored at exit, so I don't think that it causes 
troubles to use a core dump.

Dump a core file doesn't rely on process registers or mappings. Why do you 
think that it will break the core dump function?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new?PYTHONNOFAULTHANDLER?environment variable

[Alexander Belopolsky]

On Mon, Dec 20, 2010 at 12:51 PM, Victor Stinner vstin...@edenwall.com wrote:
..
 Are you sure that a signal handler changes the registers? At least Linux saves
 integer and float registers before calling the signal handler, and then
 restores them.

What if the program crashes (again) in the signal handler before
having a chance to restore registers?  Also, can you point to
documentation that describes register restoring behavior?  I am not
sure all registers are (or even can be) restored.

 The fault handler only changes some registers and write some bytes on the
 stack. The stack pointer is restored at exit, so I don't think that it causes
 troubles to use a core dump.

 Dump a core file doesn't rely on process registers or mappings. Why do you
 think that it will break the core dump function?

Because I had actual experience with that using R.  Your code may be
better, but R brings up a dialog on segfault asking whether to produce
a core dump.  Occasionally, that core dump would be completely
unusable because it would point to an unrelated point in the code.  I
don't have a specific case handy because long ago I disabled segfault
handlers in the local R installation.  (I actually use RPy which loads
R into  Python as a shared library, so the two handlers will conflict
anyways.)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER?environment variable

[R. David Murray]

On Mon, 20 Dec 2010 15:55:57 +0100, Stefan Krah ste...@bytereef.org wrote:
 Victor Stinner vstin...@edenwall.com wrote:
  Stefan Krah wrote:
   I think the output is not particularly informative:
   
   $ ./python Lib/test/crashers/gc_inspection.py
   (object object at 0x7f01827ad870, NULL, NULL, NULL, NULL, 
   NULL,
   NULL, NULL, NULL, NULL) Fatal Python error: Segmentation fault

   Traceback (most recent call first):
 File Lib/test/crashers/gc_inspection.py, line 29 in g
 File Lib/test/crashers/gc_inspection.py, line 32 in module
   Segmentation fault
   Segmentation fault
 
  The backtrace is valid. Don't you think that this backtrace is more useful
  than just Segmentation fault?
 
 Perhaps I misunderstood, but I thought the purpose of the patch was to
 let developers act more quickly on bug reports. I wonder if this output
 really speeds up the process.
 
 Do you have an example bug where this patch helps in finding the precise
 location of a segfault?

How is 'line 29 in g' not more precise than 'Segmentation fault'?

Now imagine that this error is being produced from a 20,000 line
application program bundle...  

--
R. David Murray  www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER?environment variable

[Stefan Krah]

Victor Stinner vstin...@edenwall.com wrote:
 Le lundi 20 décembre 2010 12:08:35, Stefan Krah a écrit :
  Another test is hanging indefinitely (Ubuntu 64-bit):
  
  $ ./python Lib/test/crashers/nasty_eq_vs_dict.py
  [hanging with no output]
 
 Oh, I understood. I always test with Python compiled using --with-pydebug. 
 With pydebug, a segfault occurs. Without pydebug, there is an unlimited loop.

I'll open a feature request for more reliable segfaults. ;)


Still, on FreeBSD-8.0 the patch does not produce a traceback and also
appears to change the signal:


[without the patch]
$ ./python Lib/test/crashers/compiler_recursion.py 
Segmentation fault: 11 (core dumped)
$ echo $?
139


[with the patch]
$ ./python Lib/test/crashers/compiler_recursion.py 
Illegal instruction: 4 (core dumped)
$ echo $?
132


Stefan Krah


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER?environment variable

[Martin v. Löwis]

 Do you have an example bug where this patch helps in finding the precise
 location of a segfault?
 
 How is 'line 29 in g' not more precise than 'Segmentation fault'?

I think Stefan is treating precise location as an absolute property,
not a relative one. The precise location would be the very instruction
causing the fault; this is something that a core dump provides, unless
a signal handler interferes with that.

In many crashes I fixed, the Python stack trace was irrelevant to
understand the nature of the crash. Of course, there were also counter-
examples where you'ld need the Python stack (or, rather, the entire
interpreter trace) to understand how the crashing case can occur in
the first place.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Martin v. Löwis]

 The GIL is likely held by a different thread, then.
 _PyThreadState_Current will point to the state of this other thread.
 
 I tested when the GIL released: the fault handler is unable to retrieve the 
 thread state and so the backtrace is not printed. Which thread state should 
 be 
 retrieve? I don't know yet: maybe the last active thread state?
 
 It doesn't look trivial to fix this issue because if there is no thread (only 
 the main thread), release the GIL clears the thread state pointer (to it to 
 NULL) without storing it somewhere else (whereas with multiple threads, the 
 last active thread is stored in gil_last_holder. I should ask Antoine (or 
 someone one) how the new GIL works.

I don't think this is possible to fix. Just put a sleep() call before
the crash, and have a parallel thread perform a while-loop to see what
I mean: it should give you a stack trace of the wrong thread, possibly
touching transient objects which go away while you inspect them.

 Another variable might be created to store the (pointer to the) last active 
 thread state.

Please understand: the last active thread may not be the one that got
the signal. Instead, it may be another thread that keeps running while
your thread crashes.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Martin v. Löwis]

 This is also what I think. Installing a signal handler is a fairly
 drastic action, and I don't think the code has been sufficiently
 reviewed yet.
 
 How much more review should it receive?

There should be at least one reviewer with an established track record
of being interested/knowledgable in the POSIX APIs and cross-platform
aspects of it. As I said, any of the Twisted guys would qualify.

 Well, Python already does (and also apparently messes with other things
 such as FPU state), so that's a separate issue altogether.

Not at all. Some signal handlers are more likely to interfere with the
the rest of the application than others.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Antoine Pitrou]

Le dimanche 19 décembre 2010 à 19:53 +0100, Martin v. Löwis a écrit :
  This is also what I think. Installing a signal handler is a fairly
  drastic action, and I don't think the code has been sufficiently
  reviewed yet.
  
  How much more review should it receive?
 
 There should be at least one reviewer with an established track record
 of being interested/knowledgable in the POSIX APIs and cross-platform
 aspects of it.
 As I said, any of the Twisted guys would qualify.

James Knight already posted in this thread. Is this sufficient to you?

I don't think it's reasonable to expect the Twisted guys (whoever they
are...) to come and review our code, while they struggle with their own
bug queue. It would certainly be beneficial for the stdlib to get as
much review attention as you are asking for this patch, but it is also
totally impractical to expect such a level of attention from a community
of volunteers.

If we aren't knowledgeable enough to maintain signal handling code, then
we should simply remove the signal module from the stdlib (with an
appropriate deprecation period, that is) and let interested people
maintain it as a third-party module. That would also close some
lingering issues in the tracker, and free up some of our time.

  Well, Python already does (and also apparently messes with other things
  such as FPU state), so that's a separate issue altogether.
 
 Not at all. Some signal handlers are more likely to interfere with the
 the rest of the application than others.

Can you clarify why you think those signal handlers fall into that
description? After all, SIGINT, SIGILL and friends only get triggered in
catastrophic conditions.

Thanks

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Alexander Belopolsky]

On Sun, Dec 19, 2010 at 2:20 PM, Antoine Pitrou solip...@pitrou.net wrote:
..
 There should be at least one reviewer with an established track record
 of being interested/knowledgable in the POSIX APIs and cross-platform
 aspects of it.
 As I said, any of the Twisted guys would qualify.

 James Knight already posted in this thread. Is this sufficient to you?


I cannot speak for Martin of course, but I think we should first see
if James Knight's recommendations get incorporated into the patch and
whether that will be sufficient for  James Knight to speak in favor of
applying the patch.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Victor Stinner]

Le dimanche 19 décembre 2010 à 19:53 +0100, Martin v. Löwis a écrit :
 There should be at least one reviewer with an established track record
 of being interested/knowledgable in the POSIX APIs and cross-platform
 aspects of it.

Functions used by the fault handler:
 - write()
 - PyUnicode_Check()
 - PyFrame_GetLineNumber()
 - DebugBreak() (Windows, in debug mode, only)
 - abort()
 - (macro) PyUnicode_GET_SIZE() and PyUnicode_AS_UNICODE()
 - PyUnicode_Check(), PyFrame_Check()
 - PyFrame_GetLineNumber()
 - _Py_atomic_load_relaxed()
 - _PyThreadState_GetFrame()

I suppose that all *Py*() functions are POSIX compliants. DebugBreak()
is specific to Windows. Only write() and abort() have to be POSIX
compliant. According to Linux manual pages:

 - abort() conforms to SVr4, POSIX.1-2001, 4.3BSD, C89, C99.
 - write() conforms to SVr4, 4.3BSD, POSIX.1-2001

--

Functions used by the other parts of the patch:
 - signal() or sigaction()+sigemptyset()
 - (optional) sigaltstack()
 - PyMem_Malloc(), PyMem_Free()
 - Py_GETENV()
 - (optional) getpid(), kill() (used by the tests)

signal() conforms to C89, C99, POSIX.1-2001.

So I don't see any function incompatible with POSIX.

(Don't read old versions of my patch, read the latest version)

--

About the cross-platform question: it looks like SIGFPE fault is not
catchable on Windows. I only tried Linux and Windows, but there are a
unit test for each signal and we have 60+ buildbots with various
platforms. Can't we wait for the buildbot results?

  Well, Python already does (and also apparently messes with other things
  such as FPU state), so that's a separate issue altogether.
 
 Not at all. Some signal handlers are more likely to interfere with the
 the rest of the application than others.

Python installs its own SIGINT handler. I think that a SIGINT handler is
more intrusive than a SIGSEGV or SIGILL handler.

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Victor Stinner]

Le samedi 18 décembre 2010 à 17:23 +0100, Georg Brandl a écrit :
 Well, without a closer I assume that for some crashes it's just not
 possible anymore for the Python interpreter to even print out the
 traceback?

The worst case occurs if the frame objects are corrupted, eg. if the
filename of a frame is an invalid object (eg. pointer to NULL or to a
corrupted unicode object).

If the file descriptor 2 is not stderr (eg. if there is no file
descriptor 2), the backtrace cannot be printed.

Except of these two issues, I think that the fault handler is always
able to display the backtrace, even on stack overflow or invalid memory
write.

--

Older versions of my patch were less reliable:

 - allocate memory on the heap = only use the stack (only few bytes)
 - call complex Python functions (like UTF-8 codec with
backslashreplace error handler) = use a very simple ASCII
+backslashreplace implementation and avoid complex functions
 - use not signal-safe functions like printf = use write() and simple
functions to format numbers
 - may go into an unlimited loop if there is a loop in the frame linked
list = limit the loop to MAX_DEPTH (100) iterations

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Victor Stinner]

Le samedi 18 décembre 2010 à 13:55 -0500, Alexander Belopolsky a écrit :
 I am -1 on the feature as written.  I would be -0 if it did not
 install signal handlers by default and even better was implemented in
 a separate module, not in core.

I think that the feature is useless if it is disabled by default. How do
you know that you application will crash? The idea is to give
informations to the user when an application crashs: the user can use
the backtrace or send it to the developer. Segmentation faults are
usually not (easilly) reproductible :-( So even if you enable the fault
handler, you may not be able to replay the crash. Or even worse, the
fault may not occurs at all when you enable the fault handler...
(Heisenbugs!)

 This feature is similar to the one that is implemented by R and with
 R, it was always a problem when R was embedded as a library.

Which kind of problem? A conflict with an existing fault handler?

 It has always been a welcome feature of Python that its core did 
 not mess with the application global state.

What do mess the application global state?

 Before this is approved, I would like to see a discussion of the
 consequences for embedded Python

I suppose that if a fault occurs outside Python, the fault handler
displays an irrevelent or empty Python backtrace.

The fault handler is installed by Py_InitializeEx(), and uninstalled by
Py_Finalize().

If the program has its own fault handler or doesn't want Python fault
handler, it can set PYTHONNOFAULTHANDLER environment variable. Should we
add a C function to the Python API to configure (disable) the fault
handler?

I only know *two* applications catching faults: Mplayer and aMule. I
don't think that they embed Python. If they do, I think that these *two*
applications can do something special. Since there are only two
applications having their own fault handler, I don't think that we
should disable the fault handler by default.

(System wide fault handler, like Apport on Ubuntu, can be configured
system wide by setting the PYTHONNOFAULTHANDLER environment variable)

 and particularly for the case when there are multiple interpreters 
 in the same process.

What? Is it a myth or does Python really support multiple interpreters
in the same process? How is it possible? Who uses this?

_Py_DumpBacktrace() function (of my patch) uses
_Py_atomic_load_relaxed(_PyThreadState_Current) to get the current
thread state. I picked up this instruction from PyThreadState_Get(). If
it doesn't work, PyThreadState_Get() should be fixed too.

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Scott Dial]

On 12/18/2010 8:50 AM, James Y Knight wrote:
 I think instead of calling abort() to kill the process, you should:
 - install the signal handler with SA_NODEFER|SA_RESETHAND (or if
 sigaction is not present, explicitly reset the action to SIG_DFL and
 unblock first thing upon entering the handler), and then,
 - at the end of the handler, kill(getpid(), orig_signal) in order to
 abort the process.

I concur with this being the correct way to right such a handler.

 This has two advantages: 1) the process's exit code will actually 
 show the correct signal,

It's more than an advantage: it's the only correct way to handle a
termination signal.

 2) it might let the OS fault handlers work properly
 as well -- I'm not sure. If it does, you may want to experiment with
 whether having or omitting SA_NODEFER gives a better backtrace (from the
 OS mechanism) in that case.

Even if that doesn't work, things like the grsecurity patches to linux
use these signals to detect exploits and log them and do throttling.
Calling abort() effectively converts all of these faults into SIGABRT
terminations that are considered (more?) innocent terminations.

-- 
Scott Dial
sc...@scottdial.com
scod...@cs.indiana.edu

-- 
Scott Dial
sc...@scottdial.com
scod...@cs.indiana.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Victor Stinner]

Le dimanche 19 décembre 2010 à 20:20 +0100, Antoine Pitrou a écrit :
 Can you clarify why you think those signal handlers fall into that
 description? After all, SIGINT, SIGILL and friends only get triggered in
 catastrophic conditions.

SIGSEGV, not SIGINT

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Antoine Pitrou]

On Sun, 19 Dec 2010 23:37:58 +0100
Victor Stinner victor.stin...@haypocalc.com wrote:
 Le dimanche 19 décembre 2010 à 20:20 +0100, Antoine Pitrou a écrit :
  Can you clarify why you think those signal handlers fall into that
  description? After all, SIGINT, SIGILL and friends only get triggered in
  catastrophic conditions.
 
 SIGSEGV, not SIGINT

Oops, sorry for the mistake. I hope this didn't stir any confusion
beyond my own ;)

cheers

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Victor Stinner]

Le samedi 18 décembre 2010 à 08:50 -0500, James Y Knight a écrit :
 I think instead of calling abort() to kill the process, you should:
 - install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is 
 not present, explicitly reset the action to SIG_DFL and unblock first thing 
 upon entering the handler), and then,
 - at the end of the handler, kill(getpid(), orig_signal) in order to abort 
 the process.

I don't think that kill() is available on Windows.

We may call directly the previous handler in the new handler, but it is
more difficult. It looks like the prototype of a signal handler depends
on its options. Or is it possible to always pass (signum, siginfo,
ucontext) arguments? I don't know if siginfo and ucontect are portable
or not...

Another possible approach is to restore the previous handler in the
fault handler, and don't call abort(). The fault should occur again, and
so the previous signal handler will be called.

I choosed to not call the previous handler to keep the patch simple. If
you consider that we must call the previous handler, I can work on a new
patch doing that.

 This has two advantages: 1) the process's exit code will actually show the 
 correct signal

Oh, I didn't noticed that the exit code is different.

 2) it might let the OS fault handlers work properly as well -- I'm not sure. 
 If it does, you may want to experiment with whether having or omitting 
 SA_NODEFER gives a better backtrace (from the OS mechanism) in that case.

For the OS fault handler: we should try, I am not sure that calling
another signal handler (my fault handler) keeps the process state
unchanged (especially if sigaltstk() is not available).

I don't understand the SA_NODEFER option. What does it change for this
fault handler?

 If #2 actually works, you may not even need the env var, which would be nice.

I prefer to keep it optional because we don't know yet how it interacts
exactly with existing debug tools, OS tools, embedded Python
interpreters, etc.

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Victor Stinner]

Le lundi 20 décembre 2010 à 00:02 +0100, Victor Stinner a écrit :
 Another possible approach is to restore the previous handler in the
 fault handler, and don't call abort(). The fault should occur again, and
 so the previous signal handler will be called.

I implemented this simple approach in the version 10 of my patch: the
previous signal handler is called.
http://bugs.python.org/issue8863#msg124373

On my Debian Sid, the libc fatal error message is written as expected
(eg. Segmentation fault) and the exit code is different for each fault
(eg. 132 for SIGILL). So we keep the existing behaviour, the new Python
fault handler just write more information if a fault occurs.

I suppose that it is compatible with the grsecurity thing and Apport,
but I didn't tried. Can someone test it?

I prefer to restore the previous signal handler than restoring the
*default* signal handler.

 I choosed to not call the previous handler to keep the patch simple.

It's cool: restoring the signal approach keeps the code simple, and it
looks like it works :-)

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Martin v. Löwis]

 Functions used by the fault handler:
  - write()
  - PyUnicode_Check()
  - PyFrame_GetLineNumber()
  - DebugBreak() (Windows, in debug mode, only)
  - abort()
  - (macro) PyUnicode_GET_SIZE() and PyUnicode_AS_UNICODE()
  - PyUnicode_Check(), PyFrame_Check()
  - PyFrame_GetLineNumber()
  - _Py_atomic_load_relaxed()
  - _PyThreadState_GetFrame()
 
 I suppose that all *Py*() functions are POSIX compliants.

The problem is that merely being POSIX compliant would not be enough
to allow calling a function in a signal handler. Instead, the function
*also* needs to be async-signal safe. However,

  - abort() conforms to SVr4, POSIX.1-2001, 4.3BSD, C89, C99.
  - write() conforms to SVr4, 4.3BSD, POSIX.1-2001

These two functions are indeed async-signal safe. So assuming that
none of the functions above indirectly calls a non-async-signal-safe
function, this part is fine.

Looking at your function list, my other concern is that you are calling
Python API without holding the GIL, IIUC. In particular, you are
accessing _PyThreadState_Current, which may not point to the current
thread if the current thread has released the GIL.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Martin v. Löwis]

 What? Is it a myth or does Python really support multiple interpreters
 in the same process? How is it possible? Who uses this?

[Not sure if you are joking]

There is certainly some support for multiple interpreters in Python; the
most prominent user of this feature is mod_python.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Michael Foord]

On 19/12/2010 23:02, Victor Stinner wrote:

Le samedi 18 décembre 2010 à 08:50 -0500, James Y Knight a écrit :

I think instead of calling abort() to kill the process, you should:
- install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is 
not present, explicitly reset the action to SIG_DFL and unblock first thing 
upon entering the handler), and then,
- at the end of the handler, kill(getpid(), orig_signal) in order to abort the 
process.

I don't think that kill() is available on Windows.


Thanks to Brian Curtin there is now an os.kill() on Windows, but it 
doesn't handle the full range of signals.


Michael

--

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Victor Stinner]

Le lundi 20 décembre 2010 à 02:05 +0100, Martin v. Löwis a écrit :
 The problem is that merely being POSIX compliant would not be enough
 to allow calling a function in a signal handler. Instead, the function
 *also* needs to be async-signal safe.

Yes, this issue was fixed in an older version of my patch. Starting at
version 9, the signal handler only call signal-safe functions.

 Looking at your function list, my other concern is that you are calling
 Python API without holding the GIL, IIUC. In particular, you are
 accessing _PyThreadState_Current, which may not point to the current
 thread if the current thread has released the GIL.

Ah? Where does _PyThreadState_Current point to if the GIL is not hold
when the fault handler is called? My patch changes also Py_FatalError()
to display the Python backtrace.

It looks that _PyThreadState_Current can be NULL if the GIL is released.
In this case, _Py_DumpBacktrace() just do nothing. There is also a
gil_last_holder variable: can it be used to get the thread state
(especially if the thread state was deleted)?

I don't think that it will possible the acquire the GIL in
Py_FatalError() or in the fault handler.

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Martin v. Löwis]

 Looking at your function list, my other concern is that you are calling
 Python API without holding the GIL, IIUC. In particular, you are
 accessing _PyThreadState_Current, which may not point to the current
 thread if the current thread has released the GIL.
 
 Ah? Where does _PyThreadState_Current point to if the GIL is not hold
 when the fault handler is called?

The GIL is likely held by a different thread, then.
_PyThreadState_Current will point to the state of this other thread.

 It looks that _PyThreadState_Current can be NULL if the GIL is released.
 In this case, _Py_DumpBacktrace() just do nothing. There is also a
 gil_last_holder variable: can it be used to get the thread state
 (especially if the thread state was deleted)?

Of this thread? I don't think so. gil_last_holder might also refer to
a different thread.

 I don't think that it will possible the acquire the GIL in
 Py_FatalError() or in the fault handler.

I agree.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Georg Brandl]

Am 18.12.2010 01:55, schrieb Victor Stinner:
 Hi,
 
 In 2008, I proposed a patch to raise a Python exception on SIGSEVG 
 signal. In some cases, it's possible to catch the exception, log it, and 
 continue the execution. But because in some cases, the Python internal 
 state is corrupted, the idea was rejected (see the issue #3999).
 
 Someone asked me to display the Python backtrace on SIGSEGV, instead of 
 raising an exception. I implemented this idea in issue #8863. After 9 
 versions, I think that the patch is ready for inclusion. It catchs 
 SIGSEGV, SIGFPE, SIGBUS and SIGILL signals, and also display the Python 
 backtrace on fatal errors. Because some operating systems have their own 
 fault handler (eg. Ubuntu with apport), Dave Malcolm asked me to add an 
 option disable the Python handler. That's why I added the 
 PYTHONNOFAULTHANDLER environment variable (we all love long variable 
 names!). For an OS vendor, it looks like an environment variable is more 
 practical than a command line option. Eg. Mandriva sets 
 PYTHONDONTWRITEBYTECODE for the whole system.
 
 Raymond Hettinger asked me on IRC to write an email about the new 
 environment variable, so here is the question: do you agree to add the 
 new variable?

I very much like having a traceback on (some) segmentation faults, but
it's clear there needs to be a way to turn it off.  An environment variable
seems to be the obvious choice (for the reasons you stated for
PYTHONDONTWRITEBYTECODE).

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[James Y Knight]

On Dec 17, 2010, at 7:55 PM, Victor Stinner wrote:

 Hi,
 
 In 2008, I proposed a patch to raise a Python exception on SIGSEVG signal. In 
 some cases, it's possible to catch the exception, log it, and continue the 
 execution. But because in some cases, the Python internal state is corrupted, 
 the idea was rejected (see the issue #3999).
 
 Someone asked me to display the Python backtrace on SIGSEGV, instead of 
 raising an exception. I implemented this idea in issue #8863. After 9 
 versions, I think that the patch is ready for inclusion. It catchs SIGSEGV, 
 SIGFPE, SIGBUS and SIGILL signals, and also display the Python backtrace on 
 fatal errors. Because some operating systems have their own fault handler 
 (eg. Ubuntu with apport), Dave Malcolm asked me to add an option disable the 
 Python handler. 

I think instead of calling abort() to kill the process, you should:
- install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is 
not present, explicitly reset the action to SIG_DFL and unblock first thing 
upon entering the handler), and then,
- at the end of the handler, kill(getpid(), orig_signal) in order to abort the 
process.

This has two advantages: 1) the process's exit code will actually show the 
correct signal, 2) it might let the OS fault handlers work properly as well -- 
I'm not sure. If it does, you may want to experiment with whether having or 
omitting SA_NODEFER gives a better backtrace (from the OS mechanism) in that 
case.

If #2 actually works, you may not even need the env var, which would be nice. 

James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Victor Stinner]

Le 18/12/2010 13:21, Georg Brandl a écrit :

I very much like having a traceback on (some) segmentation faults,

Why do you say some segmentation faults?

but it's clear there needs to be a way to turn it off.  An environment variable
seems to be the obvious choice (for the reasons you stated for
PYTHONDONTWRITEBYTECODE).
Amaury proposed a new function sys.setsegfaultenabled(). Such function 
can be used in a customized site module. I think that I will implement 
it in my next patch, before commiting the new feature. I suppose that we 
also need a function to get the status of the fault handler.


I just don't like the name. I would prefer sys.setfaulthandlerenabled(), 
I maybe just sys.setfaulthandler(). As this feature is specific to 
CPython, should it be protected function? So:

 * sys._setfaulthandler(bool): enable/disable the fault handler
 * sys._getfaulthandler()-bool: get the status (enabled/disabled) of 
the fault handler


Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Georg Brandl]

Am 18.12.2010 14:57, schrieb Victor Stinner:
 Le 18/12/2010 13:21, Georg Brandl a écrit :
 I very much like having a traceback on (some) segmentation faults,
 Why do you say some segmentation faults?

Well, without a closer I assume that for some crashes it's just not
possible anymore for the Python interpreter to even print out the
traceback?

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Georg Brandl]

Am 18.12.2010 14:57, schrieb Victor Stinner:
 Le 18/12/2010 13:21, Georg Brandl a écrit :
 I very much like having a traceback on (some) segmentation faults,
 Why do you say some segmentation faults?
 but it's clear there needs to be a way to turn it off.  An environment 
 variable
 seems to be the obvious choice (for the reasons you stated for
 PYTHONDONTWRITEBYTECODE).
 Amaury proposed a new function sys.setsegfaultenabled(). Such function 
 can be used in a customized site module. I think that I will implement 
 it in my next patch, before commiting the new feature. I suppose that we 
 also need a function to get the status of the fault handler.
 
 I just don't like the name. I would prefer sys.setfaulthandlerenabled(), 
 I maybe just sys.setfaulthandler(). As this feature is specific to 
 CPython, should it be protected function? So:
   * sys._setfaulthandler(bool): enable/disable the fault handler
   * sys._getfaulthandler()-bool: get the status (enabled/disabled) of 
 the fault handler

In any case, this is coming pretty late; beta 2 is scheduled for this
weekend, and even if this is something that only kicks in when all hope
is lost anyway, it is a new feature.  I should like to hear approval
from a few more devs before I will let this go into 3.2.

Georg


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Antoine Pitrou]

On Sat, 18 Dec 2010 17:50:38 +0100
Georg Brandl g.bra...@gmx.net wrote:
 Am 18.12.2010 14:57, schrieb Victor Stinner:
  Le 18/12/2010 13:21, Georg Brandl a écrit :
  I very much like having a traceback on (some) segmentation faults,
  Why do you say some segmentation faults?
  but it's clear there needs to be a way to turn it off.  An environment 
  variable
  seems to be the obvious choice (for the reasons you stated for
  PYTHONDONTWRITEBYTECODE).
  Amaury proposed a new function sys.setsegfaultenabled(). Such function 
  can be used in a customized site module. I think that I will implement 
  it in my next patch, before commiting the new feature. I suppose that we 
  also need a function to get the status of the fault handler.
  
  I just don't like the name. I would prefer sys.setfaulthandlerenabled(), 
  I maybe just sys.setfaulthandler(). As this feature is specific to 
  CPython, should it be protected function? So:
* sys._setfaulthandler(bool): enable/disable the fault handler
* sys._getfaulthandler()-bool: get the status (enabled/disabled) of 
  the fault handler
 
 In any case, this is coming pretty late; beta 2 is scheduled for this
 weekend, and even if this is something that only kicks in when all hope
 is lost anyway, it is a new feature.  I should like to hear approval
 from a few more devs before I will let this go into 3.2.

I'm +1 on Victor's patch, although I don't think the sys function is
useful at all. The env var is enough IMO.
(I would even be for backporting it to the bugfix branches if there's
no downside)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Alexander Belopolsky]

On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl g.bra...@gmx.net wrote:
..
 In any case, this is coming pretty late; beta 2 is scheduled for this
 weekend, and even if this is something that only kicks in when all hope
 is lost anyway, it is a new feature.  I should like to hear approval
 from a few more devs before I will let this go into 3.2.


I am -1 on the feature as written.  I would be -0 if it did not
install signal handlers by default and even better was implemented in
a separate module, not in core.

This feature is similar to the one that is implemented by R and with
R, it was always a problem when R was embedded as a library.  It has
always been a welcome feature of Python that its core did not mess
with the application global state.

Before this is approved, I would like to see a discussion of the
consequences for embedded Python and particularly for the case when
there are multiple interpreters in the same process.

I think it would be best to start with this feature as an library
module or even as a 3rd party add-on and see how useful it will be
found in the wild.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Martin v. Löwis]

Am 18.12.2010 19:55, schrieb Alexander Belopolsky:
 On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl g.bra...@gmx.net wrote:
 ..
 In any case, this is coming pretty late; beta 2 is scheduled for this
 weekend, and even if this is something that only kicks in when all hope
 is lost anyway, it is a new feature.  I should like to hear approval
 from a few more devs before I will let this go into 3.2.

 
 I am -1 on the feature as written.  I would be -0 if it did not
 install signal handlers by default and even better was implemented in
 a separate module, not in core.

This is also what I think. Installing a signal handler is a fairly
drastic action, and I don't think the code has been sufficiently
reviewed yet.

So I'd rather see this deferred until after 3.2, to allow for additional
reviews (e.g. if some of the Twisted guys would approve it - that would
be convincing).

I also share Alexander's concern that Python just shouldn't mess with
signal handlers at all, ideally. So some trade-off has to be found to
address that concern (e.g. by making the signal handlers only active
for the executable interpreter, but not in the embedded case).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable

[Antoine Pitrou]

On Sat, 18 Dec 2010 20:36:50 +0100
Martin v. Löwis mar...@v.loewis.de wrote:
 Am 18.12.2010 19:55, schrieb Alexander Belopolsky:
  On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl g.bra...@gmx.net wrote:
  ..
  In any case, this is coming pretty late; beta 2 is scheduled for this
  weekend, and even if this is something that only kicks in when all hope
  is lost anyway, it is a new feature.  I should like to hear approval
  from a few more devs before I will let this go into 3.2.
 
  
  I am -1 on the feature as written.  I would be -0 if it did not
  install signal handlers by default and even better was implemented in
  a separate module, not in core.
 
 This is also what I think. Installing a signal handler is a fairly
 drastic action, and I don't think the code has been sufficiently
 reviewed yet.

How much more review should it receive?

 I also share Alexander's concern that Python just shouldn't mess with
 signal handlers at all, ideally. So some trade-off has to be found to
 address that concern (e.g. by making the signal handlers only active
 for the executable interpreter, but not in the embedded case).

Well, Python already does (and also apparently messes with other things
such as FPU state), so that's a separate issue altogether.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com