Re: [Python-Dev] Fault handler updated, now disabled by default
On 25 Dec, 10:31 pm, mer...@netwok.org wrote: faulthandler is a module: enable the handler is simple as import faulthandler. That sounds like a source of unwanted behavior (aka problems) if the handler is enabled by 1Cpydoc faulthandler 1D or by a pkgutil walk. You may want to consider using a function to enable the functionality (and add one to disable it). Enormous +1. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
Le dimanche 26 décembre 2010 à 14:10 +, exar...@twistedmatrix.com a écrit : On 25 Dec, 10:31 pm, mer...@netwok.org wrote: faulthandler is a module: enable the handler is simple as import faulthandler. That sounds like a source of unwanted behavior (aka problems) if the handler is enabled by 1Cpydoc faulthandler 1D or by a pkgutil walk. You may want to consider using a function to enable the functionality (and add one to disable it). Enormous +1. I don't know pkgutil. How does it work? In which case would it load the faulthandler module? faulthandler is currently only written in C. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
On Mon, Dec 27, 2010 at 8:57 AM, Victor Stinner victor.stin...@haypocalc.com wrote: Le dimanche 26 décembre 2010 à 14:10 +, exar...@twistedmatrix.com a écrit : On 25 Dec, 10:31 pm, mer...@netwok.org wrote: faulthandler is a module: enable the handler is simple as import faulthandler. That sounds like a source of unwanted behavior (aka problems) if the handler is enabled by 1Cpydoc faulthandler 1D or by a pkgutil walk. You may want to consider using a function to enable the functionality (and add one to disable it). Enormous +1. I don't know pkgutil. How does it work? In which case would it load the faulthandler module? faulthandler is currently only written in C. pkgutil includes a function that lets you walk the entire module heirarchy, implicitly importing everything, including all the builtin modules. It's one of the reasons doing things as side-effects of import is considered highly undesirable. The pydoc tests do this when they bring the (docstring-based) documentation server up to check its handling of HTTP requests. (we recently picked up an implicit addition of a logging handler by concurrent.futures due to this effect). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
Le vendredi 24 décembre 2010 à 12:59 +, Michael Foord a écrit : I hope you will include it in 3.3 though; it is great functionality. I don't know, Python 3.3 is not planned yet. Anyway, faulthandler is already available and you can use it on Python 2.5 through 3.2. I would really like to see it enabled by default as well. For different reasons, it is not a good idea to enable it by default (see the recent discussion on this list). It seemed from the discussion that the biggest barrier to enabling it by default was possible difficulties when ... Yes, that's why it was disabled by default in the latest version of my patch. (multiple interpreters, I think that my fault handler works with multiple interpreters. potential conflicts with application signal handling) Very few programs have an handler for SIGSEGV, SIGFPE, SIGILL or SIGBUS signals (but yes, there are such programs/libraries like RPy or XEmacs). A public C-API to disable the functionality per interpreter would be one option for this. Another possibility would be providing a C-API to enable it and have the Python interpreter application call this, so that the functionality remains off by default for embedded interpreters but on for normal uses. I think that it's better to disable it by default, and only enabled by the user or by the developer of a specific application. -- Anyway, you don't have to wait Python 3.3 (or Python 3.2): you can already use faulthandler ;-) I created Windows installer for Python 2.6, 2.7 and 3.1 (I will do it for 2.5 too in the next faulthandler version, faulthandler 1.0 tests don't work with Python 2.5). faulthandler is a module: enable the handler is simple as import faulthandler. Said differently, it's also disabled by default, and only enabled by the application developer. It's harder to enable it for the user, the site module should maybe be patched to load the module if an environment variable exist or a command line argument. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
faulthandler is a module: enable the handler is simple as import faulthandler. That sounds like a source of unwanted behavior (aka problems) if the handler is enabled by “pydoc faulthandler” or by a pkgutil walk. You may want to consider using a function to enable the functionality (and add one to disable it). Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
On 24/12/2010 02:21, Victor Stinner wrote: Le jeudi 23 décembre 2010 à 21:59 +0100, Georg Brandl a écrit : this thread showed that it is not at all obvious how the feature should look like Ok, I understand. I closed #8863 (rejected) and I created a third party module on the Python cheese shop: http://pypi.python.org/pypi/faulthandler/ The module works on Linux, FreeBSD and Windows with Python 2.6 through 3.2. A third party module can evolve faster outside Python. I hope you will include it in 3.3 though; it is great functionality. I would really like to see it enabled by default as well. It seemed from the discussion that the biggest barrier to enabling it by default was possible difficulties when embedding Python (multiple interpreters, potential conflicts with application signal handling). A public C-API to disable the functionality per interpreter would be one option for this. Another possibility would be providing a C-API to enable it and have the Python interpreter application call this, so that the functionality remains off by default for embedded interpreters but on for normal uses. All the best, Michael Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
Michael Foord writes: It seemed from the discussion that the biggest barrier to enabling it by default was possible difficulties when embedding Python (multiple interpreters, potential conflicts with application signal handling). A public C-API to disable the functionality per interpreter would be one option for this. That's not really good enough. The point of installing a handler like this is to catch them squirmers. All you have to do is override some incautious developer's own squirmer-trap handler once, and Python has made an Enemy-For-Life. (This happened to XEmacs with esound. I immediately removed esound and anything that depends on it from my workstation. ;-) YMMV and you may think that that is not so important; my point is that the proposal to provide a way to disable does not at all address the objection. Another possibility would be providing a C-API to enable it and have the Python interpreter application call this, so that the functionality remains off by default for embedded interpreters but on for normal uses. I think this is heading in the right direction. Note: My own experience with such handlers has been positive, but it does not involve embedding interpreters in either direction, so not really helpful in addressing this objection. Precisely *because* my own experience has been positive, I worry about interfering with some third party's handler. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
Le jeudi 23 décembre 2010 à 03:37 +0100, Martin v. Löwis a écrit : So, do you agree with the fault handler? Does someone want to give a last review because I commit it? It's a new feature, so regardless of whether it's correct or not (which I haven't reviewed yet), I don't think it should go in before 3.2 is released. I tested my patch on Windows (XP), Ubuntu (10.4) and FreeBSD (8) and it works correctly: all tests pass and the system fault handler (Windows popup, Apport and core dump) is also called. The fault handler is disabled by default and it is clearly separated (eg. it doesn't touch the API of a module). Can't you make an exception for this new feature? The first answer to my previous email thread was Georg, answer starting with I very much like having a traceback on (some) segmentation faults :-) Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
The fault handler is disabled by default and it is clearly separated (eg. it doesn't touch the API of a module). Can't you make an exception for this new feature? Ultimately, it's for the release manager to decide, so I don't need to make an exception. However, I think that special cases aren't special enough to break the rules. I still wish that the beta releases had been deferred until after the Mercurial switchover, but alas, the release manager has decided otherwise - explicitly pointing out that the rationale for releasing beta 1 was to stop accepting new features. The motivation for freezing features before feature releases is not so much that existing applications may be affected by API changes (3.2 will change APIs, anyway, and users fully expect to need some porting), but that the features that are about to be released need some time for testing. This specific feature has seen very little testing. Giving it a full release cycle (i.e. until 3.3) would somewhat reduce the need for a more careful code review. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
Am 23.12.2010 19:23, schrieb Martin v. Löwis: The fault handler is disabled by default and it is clearly separated (eg. it doesn't touch the API of a module). Can't you make an exception for this new feature? Ultimately, it's for the release manager to decide, so I don't need to make an exception. However, I think that special cases aren't special enough to break the rules. I still wish that the beta releases had been deferred until after the Mercurial switchover, but alas, the release manager has decided otherwise - explicitly pointing out that the rationale for releasing beta 1 was to stop accepting new features. The main rationale was that the Mercurial switchover is not a fixed target and depends very much on volunteer effort, so in effect this would have been akin to deferring 3.2 for an indefinite period. The motivation for freezing features before feature releases is not so much that existing applications may be affected by API changes (3.2 will change APIs, anyway, and users fully expect to need some porting), but that the features that are about to be released need some time for testing. This specific feature has seen very little testing. Giving it a full release cycle (i.e. until 3.3) would somewhat reduce the need for a more careful code review. Exactly. I did say I like the feature, but that was a) before beta 2 was released, now the next release is a release candidate, and b) this thread showed that it is not at all obvious how the feature should look like. The fact that it isn't enabled by default also makes it seem less attractive to me, but I understand the reasons why it shouldn't be on by default. Therefore, I'm not willing to make an exception here. Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
On 23/12/2010, Victor Stinner victor.stin...@haypocalc.com wrote: I tested my patch on Windows (XP), Ubuntu (10.4) and FreeBSD (8) and it works correctly: all tests pass and the system fault handler (Windows popup, Apport and core dump) is also called. Doesn't build for me without #ifdef HAVE_UNISTD_H in Python/fault.c and you missed my comment about raise vs. kill in the other thread which lets the SIGILL test run as well. With those changes, I get: test_disabled (test.test_fault.FaultHandlerTests) ... ok test_fatal_error (test.test_fault.FaultHandlerTests) ... FAIL test_gil_released (test.test_fault.FaultHandlerTests) ... ok test_sigbus (test.test_fault.FaultHandlerTests) ... skipped 'need _testcapi.sigbus()' test_sigfpe (test.test_fault.FaultHandlerTests) ... skipped 'SIGFPE cannot be caught on Windows' test_sigill (test.test_fault.FaultHandlerTests) ... ok test_sigsegv (test.test_fault.FaultHandlerTests) ... ok test_xoption (test.test_fault.FaultHandlerTests) ... ok So, this does basically work as there is a SIGSEGV compatibility hack in VS8 and later (I'd neglected to compare the language across the different MSDN pages). The failure is due to the test not expecting an extra note the assert gives at the end. See attached interdiff for suggested changes. Martin issue8863_incr.diff Description: Binary data ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
Le jeudi 23 décembre 2010 à 21:59 +0100, Georg Brandl a écrit : this thread showed that it is not at all obvious how the feature should look like Ok, I understand. I closed #8863 (rejected) and I created a third party module on the Python cheese shop: http://pypi.python.org/pypi/faulthandler/ The module works on Linux, FreeBSD and Windows with Python 2.6 through 3.2. A third party module can evolve faster outside Python. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
Le jeudi 23 décembre 2010 à 22:58 +, Martin (gzlist) a écrit : On 23/12/2010, Victor Stinner victor.stin...@haypocalc.com wrote: I tested my patch on Windows (XP), Ubuntu (10.4) and FreeBSD (8) and it works correctly: all tests pass and the system fault handler (Windows popup, Apport and core dump) is also called. Doesn't build for me without #ifdef HAVE_UNISTD_H in Python/fault.c Yes, I wrote a comment about that in the issue. But then I realized that this header is not needed at all. you missed my comment about raise vs. kill in the other thread which lets the SIGILL test run as well Oh, I didn't noticed that this change enables sigill() on Windows. Nice :-) I created a third party module from my patch: http://pypi.python.org/pypi/faulthandler/ I commited your patch, thanks. I added you to the authors as a contributor. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
Am 24.12.2010 03:21, schrieb Victor Stinner: Le jeudi 23 décembre 2010 à 21:59 +0100, Georg Brandl a écrit : this thread showed that it is not at all obvious how the feature should look like Ok, I understand. I closed #8863 (rejected) and I created a third party module on the Python cheese shop: http://pypi.python.org/pypi/faulthandler/ The module works on Linux, FreeBSD and Windows with Python 2.6 through 3.2. A third party module can evolve faster outside Python. Very good. I take it you'll make a new issue then for inclusion in 3.3? Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Fault handler updated, now disabled by default
Hi, Thanks to all your useful remarks, I improved my patch twice (version 10 and 11). I think that the last version (11) addresses all reported issues. The most notable change is that the fault handler is now disabled by default. It fixes also the function getting the thread state, it now works in all cases (especially if the thread causing the fault doesn't hold the GIL). Summary of the patch (#8863): - Add an optional fault handler for SIGSEGV, SIGFPE, SIGILL and SIGBUS signals displaying the Python backtrace - Setting PYTHONFAULTHANDLER environment variable, or using -X faulthandler command line option, enables it - If it's enabled, display also the Python backtrace on a fatal error Use case: when a program crashs, the user reruns its application with the fault handler enabled and tries to reproduce the crash. He/She can send the Python backtrace to the developer, or use it directly (if he/she understands it). I think that this patch helps to isolate crashs in Python or libraries, especially if the user doesn't have all tools needed to get a Python backtrace (on a crash) and the developer is unable to reproduce the bug (because it doesn't have the same OS or libraries). For example, gdb 7 and python-gdb.py are rare on Windows :-) (even on Linux!) So, do you agree with the fault handler? Does someone want to give a last review because I commit it? http://bugs.python.org/issue8863 Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
So, do you agree with the fault handler? Does someone want to give a last review because I commit it? It's a new feature, so regardless of whether it's correct or not (which I haven't reviewed yet), I don't think it should go in before 3.2 is released. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
