[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 3d55aa9e7365af76e40680271328deece27a7339 by Victor Stinner in 
branch 'master':
bpo-42923: Fix _Py_DumpExtensionModules() for NULL sysdict (GH-25262)
https://github.com/python/cpython/commit/3d55aa9e7365af76e40680271328deece27a7339


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +23998
pull_request: https://github.com/python/cpython/pull/25262

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-42955 "Add sys.module_names: list of stdlib module names (Python 
and extension modules)".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:

More "real world" example using cinder which imports 220 modules (ctypes is 
used to simulate a crash):
---
$ ./python -m venv env
$ env/bin/python -m pip install wheel
$ env/bin/python -m pip install cinder
$ env/bin/python -X dev -c 'import cinder, ctypes, sys; 
print(f"{len(sys.modules)=}"); print(); ctypes.string_at(0)'
(...)
len(sys.modules)=220

Fatal Python error: Segmentation fault

Current thread 0x7fc4a88e0740 (most recent call first):
  File "/home/vstinner/python/master/Lib/ctypes/__init__.py", line 517 in 
string_at
  File "", line 1 in 

Extension modules: greenlet._greenlet, __original_module__thread, 
__original_module_select, __original_module_time, _cffi_backend (total: 5)
Erreur de segmentation (core dumped)
---

cinder only uses 2 third party extension modules (on a total of 220 modules): 
greenlet and cffi.

Note: __original_xxx modules are aliases of stdlib modules created by eventlet 
monkey patching.

So if cinder does crash, I suggest to look at Python, but *also* look at these 
2 extensions ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 66f77caca39ba39ebe1e4a95dba6d19b20d51951 by Victor Stinner in 
branch 'master':
bpo-42923: _Py_DumpExtensionModules() ignores stdlib ext (GH-24254)
https://github.com/python/cpython/commit/66f77caca39ba39ebe1e4a95dba6d19b20d51951


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +23076
pull_request: https://github.com/python/cpython/pull/24238

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +23074
pull_request: https://github.com/python/cpython/pull/24254

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:

I created https://bugs.python.org/issue42955 to add sys.modules_names tuple: 
names of stdlib modules. Once it will be merged, I will updated this PR to 
filter the list of modules (ignore stdlib modules).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 250035d134ad482e724f73ce654682254b513ee0 by Victor Stinner in 
branch 'master':
bpo-42923: Dump extension modules on fatal error (GH-24207)
https://github.com/python/cpython/commit/250035d134ad482e724f73ce654682254b513ee0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 314b8787e0c50985ba708034b84ff5b37a1d47de by Victor Stinner in 
branch 'master':
bpo-42923: Py_FatalError() avoids fprintf() (GH-24242)
https://github.com/python/cpython/commit/314b8787e0c50985ba708034b84ff5b37a1d47de


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:


New changeset e232025025c8bd07c1d1b12a583a80c4a673f077 by Victor Stinner in 
branch 'master':
bpo-42923: Add Py_FatalError() test in test_capi (GH-24240)
https://github.com/python/cpython/commit/e232025025c8bd07c1d1b12a583a80c4a673f077


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +23064
pull_request: https://github.com/python/cpython/pull/24242

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +23062
pull_request: https://github.com/python/cpython/pull/24240

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:

Example of a Pillow crash in Python 3.10:
https://mail.python.org/archives/list/python-...@python.org/message/5DCR2R6POCOHTPFUOY4F5G7GTBYX6OPO/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:

> It's common that Python crashes are coming from third party extension 
> modules, rather than Python itself. Recent example: bpo-42891 crash (...)

Another recent example: bpo-42879 "python: Objects/abstract.c:155: 
PyObject_GetItem: Assertion `(item != NULL) ^ (PyErr_Occurred() != NULL)' 
failed." was a bug in pygame.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

STINNER Victor  added the comment:

Example using PR 24207, the new part the "Extension modules:" list at the end:
---
$ ./python -X faulthandler
Python 3.10.0a4+ (heads/master-dirty:2396614b89, Jan 13 2021, 12:09:15) 
>>> import ctypes
>>> ctypes.string_at(0)
Fatal Python error: Segmentation fault

Current thread 0x7f0aabd09740 (most recent call first):
  File "/home/vstinner/python/master/Lib/ctypes/__init__.py", line 517 in 
string_at
  File "", line 1 in 

Extension modules:
 * sys
 * builtins
 * _imp
 * _thread
 * _warnings
 * _weakref
 * _io
 * marshal
 * posix
 * time
 * faulthandler
 * _codecs
 * _signal
 * _abc
 * _stat
 * readline
 * atexit
 * itertools
 * _operator
 * _collections
 * _functools
 * _opcode
 * _sre
 * _locale
 * _ctypes
 * _struct
---

Here you can see that the _ctypes module is loaded.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +23034
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24207

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42923] Py_FatalError(): dump the list of extension modules

[STINNER Victor]

New submission from STINNER Victor :

When Python cannot report an error and its is not possible to recover from the 
error, Py_FatalError() displays an error message and exit immediately Python.

It's common that Python crashes are coming from third party extension modules, 
rather than Python itself. Recent example: bpo-42891 crash reported in Python, 
but it was a bug in lsm-db third party extension module.

I propose to enhance Py_FatalError() to attempt to dump the list of extension 
modules, to ease debugging.

I'm working on a PR.

--
components: Interpreter Core
messages: 385013
nosy: vstinner
priority: normal
severity: normal
status: open
title: Py_FatalError(): dump the list of extension modules
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com