On 20.06.2015 09:30, Victor Stinner wrote:
> Hi,
>
> I didn't get much feedback on this PEP. Since the Python 3.6 branch is
> open (default), it's probably better to push such change in the
> beginning of the 3.6 cycle, to catch issues earlier.
>
> Are you ok to chain exceptions at C level by default?
I think it's a good idea to make C APIs available to
simplify chaining exceptions at the C level, but don't
believe that always doing this by default is a good idea.
It should really be a case-by-case decision, IMO.
Note that Python exceptions are cheap to raise in C
(very much unlike in Python), so making this more
expensive by default would introduce a significant
overhead - without much proven benefit.
More below...
> PEP: 490
> Title: Chain exceptions at C level
> Version: $Revision$
> Last-Modified: $Date$
> Author: Victor Stinner <victor.stin...@gmail.com>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 25-March-2015
> Python-Version: 3.6
>
>
> Abstract
> ========
>
> Chain exceptions at C level, as already done at Python level.
>
>
> Rationale
> =========
>
> Python 3 introduced a new killer feature: exceptions are chained by
> default, PEP 3134.
>
> Example::
>
> try:
> raise TypeError("err1")
> except TypeError:
> raise ValueError("err2")
>
> Output::
>
> Traceback (most recent call last):
> File "test.py", line 2, in <module>
> raise TypeError("err1")
> TypeError: err1
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "test.py", line 4, in <module>
> raise ValueError("err2")
> ValueError: err2
>
> Exceptions are chained by default in Python code, but not in
> extensions written in C.
>
> A new private ``_PyErr_ChainExceptions()`` function was introduced in
> Python 3.4.3 and 3.5 to chain exceptions. Currently, it must be called
> explicitly to chain exceptions and its usage is not trivial.
>
> Example of ``_PyErr_ChainExceptions()`` usage from the ``zipimport``
> module to chain the previous ``OSError`` to a new ``ZipImportError``
> exception::
>
> PyObject *exc, *val, *tb;
> PyErr_Fetch(&exc, &val, &tb);
> PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
> _PyErr_ChainExceptions(exc, val, tb);
>
> This PEP proposes to also chain exceptions automatically at C level to
> stay consistent and give more information on failures to help
> debugging. The previous example becomes simply::
>
> PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
>
>
> Proposal
> ========
>
> Modify PyErr_*() functions to chain exceptions
> ----------------------------------------------
>
> Modify C functions raising exceptions of the Python C API to
> automatically chain exceptions: modify ``PyErr_SetString()``,
> ``PyErr_Format()``, ``PyErr_SetNone()``, etc.
>
>
> Modify functions to not chain exceptions
> ----------------------------------------
>
> Keeping the previous exception is not always interesting when the new
> exception contains information of the previous exception or even more
> information, especially when the two exceptions have the same type.
>
> Example of an useless exception chain with ``int(str)``::
>
> TypeError: a bytes-like object is required, not 'type'
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> TypeError: int() argument must be a string, a bytes-like object or
> a number, not 'type'
>
> The new ``TypeError`` exception contains more information than the
> previous exception. The previous exception should be hidden.
>
> The ``PyErr_Clear()`` function can be called to clear the current
> exception before raising a new exception, to not chain the current
> exception with a new exception.
>
>
> Modify functions to chain exceptions
> ------------------------------------
>
> Some functions save and then restore the current exception. If a new
> exception is raised, the exception is currently displayed into
> sys.stderr or ignored depending on the function. Some of these
> functions should be modified to chain exceptions instead.
>
> Examples of function ignoring the new exception(s):
>
> * ``ptrace_enter_call()``: ignore exception
> * ``subprocess_fork_exec()``: ignore exception raised by enable_gc()
> * ``t_bootstrap()`` of the ``_thread`` module: ignore exception raised
> by trying to display the bootstrap function to ``sys.stderr``
> * ``PyDict_GetItem()``, ``_PyDict_GetItem_KnownHash()``: ignore
> exception raised by looking for a key in the dictionary
> * ``_PyErr_TrySetFromCause()``: ignore exception
> * ``PyFrame_LocalsToFast()``: ignore exception raised by
> ``dict_to_map()``
> * ``_PyObject_Dump()``: ignore exception. ``_PyObject_Dump()`` is used
> to debug, to inspect a running process, it should not modify the
> Python state.
> * ``Py_ReprLeave()``: ignore exception "because there is no way to
> report them"
> * ``type_dealloc()``: ignore exception raised by
> ``remove_all_subclasses()``
> * ``PyObject_ClearWeakRefs()``: ignore exception?
> * ``call_exc_trace()``, ``call_trace_protected()``: ignore exception
> * ``remove_importlib_frames()``: ignore exception
> * ``do_mktuple()``, helper used by ``Py_BuildValue()`` for example:
> ignore exception?
> * ``flush_io()``: ignore exception
> * ``sys_write()``, ``sys_format()``: ignore exception
> * ``_PyTraceback_Add()``: ignore exception
> * ``PyTraceBack_Print()``: ignore exception
Which of these do you think would benefit from chaining exceptions ?
> Examples of function displaying the new exception to ``sys.stderr``:
>
> * ``atexit_callfuncs()``: display exceptions with
> ``PyErr_Display()`` and return the latest exception, the function
> calls multiple callbacks and only returns the latest exception
> * ``sock_dealloc()``: log the ``ResourceWarning`` exception with
> ``PyErr_WriteUnraisable()``
> * ``slot_tp_del()``: display exception with
> ``PyErr_WriteUnraisable()``
> * ``_PyGen_Finalize()``: display ``gen_close()`` exception with
> ``PyErr_WriteUnraisable()``
> * ``slot_tp_finalize()``: display exception raised by the
> ``__del__()`` method with ``PyErr_WriteUnraisable()``
> * ``PyErr_GivenExceptionMatches()``: display exception raised by
> ``PyType_IsSubtype()`` with ``PyErr_WriteUnraisable()``
Same here.
In many cases, there's no way to report exceptions at all,
so chaining them make things better :-)
> Backward compatibility
> ======================
>
> A side effect of chaining exceptions is that exceptions store
> traceback objects which store frame objects which store local
> variables. Local variables are kept alive by exceptions. A common
> issue is a reference cycle between local variables and exceptions: an
> exception is stored in a local variable and the frame indirectly
> stored in the exception. The cycle only impacts applications storing
> exceptions.
It's not only about reference cycles. Tracebacks can also keep
files, sockets and other external resources open as well as
prevent large memory areas from being freed.
> The reference cycle can now be fixed with the new
> ``traceback.TracebackException`` object introduced in Python 3.5. It
> stores informations required to format a full textual traceback without
> storing local variables.
>
> The ``asyncio`` is impacted by the reference cycle issue. This module
> is also maintained outside Python standard library to release a
> version for Python 3.3. ``traceback.TracebackException`` will maybe
> be backported in a private ``asyncio`` module to fix reference cycle
> issues.
>
>
> Alternatives
> ============
>
> No change
> ---------
>
> A new private ``_PyErr_ChainExceptions()`` function is enough to chain
> manually exceptions.
>
> Exceptions will only be chained explicitly where it makes sense.
>
>
> New helpers to chain exceptions
> -------------------------------
>
> Functions like ``PyErr_SetString()`` don't chain automatically
> exceptions. To make the usage of ``_PyErr_ChainExceptions()`` easier,
> new private functions are added:
>
> * ``_PyErr_SetStringChain(exc_type, message)``
> * ``_PyErr_FormatChain(exc_type, format, ...)``
> * ``_PyErr_SetNoneChain(exc_type)``
> * ``_PyErr_SetObjectChain(exc_type, exc_value)``
>
> Helper functions to raise specific exceptions like
> ``_PyErr_SetKeyError(key)`` or ``PyErr_SetImportError(message, name,
> path)`` don't chain exceptions. The generic
> ``_PyErr_ChainExceptions(exc_type, exc_value, exc_tb)`` should be used
> to chain exceptions with these helper functions.
>
>
> Appendix
> ========
>
> PEPs
> ----
>
> * `PEP 3134 -- Exception Chaining and Embedded Tracebacks
> <https://www.python.org/dev/peps/pep-3134/>`_ (Python 3.0):
> new ``__context__`` and ``__cause__`` attributes for exceptions
> * `PEP 415 - Implement context suppression with exception attributes
> <https://www.python.org/dev/peps/pep-0415/>`_ (Python 3.3):
> ``raise exc from None``
> * `PEP 409 - Suppressing exception context
> <https://www.python.org/dev/peps/pep-0409/>`_
> (superseded by the PEP 415)
>
>
> Python C API
> ------------
>
> The header file ``Include/pyerror.h`` declares functions related to
> exceptions.
>
> Functions raising exceptions:
>
> * ``PyErr_SetNone(exc_type)``
> * ``PyErr_SetObject(exc_type, exc_value)``
> * ``PyErr_SetString(exc_type, message)``
> * ``PyErr_Format(exc, format, ...)``
>
> Helpers to raise specific exceptions:
>
> * ``PyErr_BadArgument()``
> * ``PyErr_BadInternalCall()``
> * ``PyErr_NoMemory()``
> * ``PyErr_SetFromErrno(exc)``
> * ``PyErr_SetFromWindowsErr(err)``
> * ``PyErr_SetImportError(message, name, path)``
> * ``_PyErr_SetKeyError(key)``
> * ``_PyErr_TrySetFromCause(prefix_format, ...)``
>
> Manage the current exception:
>
> * ``PyErr_Clear()``: clear the current exception,
> like ``except: pass``
> * ``PyErr_Fetch(exc_type, exc_value, exc_tb)``
> * ``PyErr_Restore(exc_type, exc_value, exc_tb)``
> * ``PyErr_GetExcInfo(exc_type, exc_value, exc_tb)``
> * ``PyErr_SetExcInfo(exc_type, exc_value, exc_tb)``
>
> Others function to handle exceptions:
>
> * ``PyErr_ExceptionMatches(exc)``: check to implement
> ``except exc: ...``
> * ``PyErr_GivenExceptionMatches(exc1, exc2)``
> * ``PyErr_NormalizeException(exc_type, exc_value, exc_tb)``
> * ``_PyErr_ChainExceptions(exc_type, exc_value, exc_tb)``
>
>
> Python Issues
> -------------
>
> Chain exceptions:
>
> * `Issue #23763: Chain exceptions in C
> <http://bugs.python.org/issue23763>`_
> * `Issue #23696: zipimport: chain ImportError to OSError
> <http://bugs.python.org/issue23696>`_
> * `Issue #21715: Chaining exceptions at C level
> <http://bugs.python.org/issue21715>`_: added
> ``_PyErr_ChainExceptions()``
> * `Issue #18488: sqlite: finalize() method of user function may be
> called with an exception set if a call to step() method failed
> <http://bugs.python.org/issue18488>`_
> * `Issue #23781: Add private _PyErr_ReplaceException() in 2.7
> <http://bugs.python.org/issue23781>`_
> * `Issue #23782: Leak in _PyTraceback_Add
> <http://bugs.python.org/issue23782>`_
>
> Changes preventing to loose exceptions:
>
> * `Issue #23571: Raise SystemError if a function returns a result with an
> exception set <http://bugs.python.org/issue23571>`_
> * `Issue #18408: Fixes crashes found by pyfailmalloc
> <http://bugs.python.org/issue18408>`_
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
>
>
> ..
> Local Variables:
> mode: indented-text
> indent-tabs-mode: nil
> sentence-end-double-space: t
> fill-column: 70
> coding: utf-8
> End:
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/mal%40egenix.com
>
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jun 20 2015)
>>> Python Projects, Coaching and Consulting ... http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
2015-06-16: Released eGenix pyOpenSSL 0.13.10 ... http://egenix.com/go78
2015-06-10: Released mxODBC Plone/Zope DA 2.2.2 http://egenix.com/go76
2015-07-20: EuroPython 2015, Bilbao, Spain ... 30 days to go
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
