STINNER Victor <vstin...@redhat.com> added the comment:
With an additonal change on SOABI (I will open a separated issue for that), PR
12946 allows to load lxml built in release mode in a Python built in debug
mode! That's *very* useful for debugging: see my gdb example below.
---
I just modified the ABI of debug build so release and debug build now have the
same ABI: bpo-36465.
I wrote a patch to use the same sys.implementation.cache_tag (SOABI) in release
and debug mode:
diff --git a/configure b/configure
index b02d17c053..38eb7f1bd6 100755
--- a/configure
+++ b/configure
@@ -6325,7 +6325,6 @@ $as_echo "#define Py_DEBUG 1" >>confdefs.h
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
$as_echo "yes" >&6; };
Py_DEBUG='true'
- ABIFLAGS="${ABIFLAGS}d"
else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
$as_echo "no" >&6; }; Py_DEBUG='false'
fi
diff --git a/configure.ac b/configure.ac
index 65d3f8e691..1b2cd3076c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1223,7 +1223,6 @@ then
[Define if you want to build an interpreter with many run-time checks.])
AC_MSG_RESULT(yes);
Py_DEBUG='true'
- ABIFLAGS="${ABIFLAGS}d"
else AC_MSG_RESULT(no); Py_DEBUG='false'
fi],
[AC_MSG_RESULT(no)])
(That's a temporary patch, I will design a better solution later.)
Using this patch + PR 12946, it becomes possible to load a C extension compiled
in release mode in a debug Python!
---
Example building lxml in release mode and then load it in debug mode.
Install Python in *release* mode into /opt/py38release (shared libpython):
git clean -fdx; ./configure --enable-shared --prefix /opt/py38release && make
&& make install
Install Python in *debug* mode into /opt/py38debug (shared libpython):
git clean -fdx; ./configure CFLAGS="-O0" --enable-shared --prefix
/opt/py38debug --with-pydebug && make && make install
Build lxml in release mode:
LD_LIBRARY_PATH=/opt/py38release/lib/ /opt/py38release/bin/python3.8 -m pip
install lxml
By default, the debug Python doesn't have lxml:
$ LD_LIBRARY_PATH=/opt/py38debug/lib/ /opt/py38debug/bin/python3.8 -c 'import
lxml'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'lxml'
Give access to C extensions compiled in release mode to the debug Python:
$ LD_LIBRARY_PATH=/opt/py38debug/lib/
PYTHONPATH=/opt/py38release/lib/python3.8/site-packages
/opt/py38debug/bin/python3.8
Python 3.8.0a3+ (heads/omit_libpython-dirty:8a03782387, Apr 25 2019, 02:52:01)
>>> import lxml
>>> import lxml.etree
>>> lxml.etree
<module 'lxml.etree' from
'/opt/py38release/lib/python3.8/site-packages/lxml/etree.cpython-38-x86_64-linux-gnu.so'>
>>> import sys
>>> sys.gettotalrefcount()
108304
It works!
Explanation:
* This is a debug build: sys.gettotalrefcount() is present (and works)
* lxml has been compiled in release mode
* etree.cpython-38-x86_64-linux-gnu.so is not linked to libpython and so
doesn't rely on the *release* /opt/py38release/lib/libpython3.8.so
* this lxml can be loaded in a Python compiled in debug mode!
---
Now let's have a look at the gdb experience of release vs debug Python build.
I put a breakpoint on lxml.etree.iterparse('example.xml'): the C function is
called __pyx_tp_new_4lxml_5etree_iterparse.
Using the release build, gdb fails to read many C local variables:
$ LD_LIBRARY_PATH=/opt/py38release/lib/ gdb -args
/opt/py38release/bin/python3.8 parse.py
(gdb) source /home/vstinner/prog/python/master/python-gdb.py
(gdb) b __pyx_tp_new_4lxml_5etree_iterparse
Function "__pyx_tp_new_4lxml_5etree_iterparse" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
(gdb) run
Breakpoint 1, __pyx_tp_new_4lxml_5etree_iterparse (t=0x7fffea724900
<__pyx_type_4lxml_5etree_iterparse>, a=('example.xml',), k=0x0) at
src/lxml/etree.c:218930
218930 src/lxml/etree.c: No such file or directory.
(gdb) py-bt
Traceback (most recent call first):
File "parse.py", line 4, in func
context = etree.iterparse('example.xml')
File "parse.py", line 12, in <module>
func("arg")
(gdb) frame 4
#4 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at
Python/ceval.c:3268
3268 res = call_function(&sp, oparg, NULL);
(gdb) p f
$1 = <optimized out>
(gdb) p co
$2 = <optimized out>
(gdb) p sp
$3 = <optimized out>
(gdb) p oparg
$4 = <optimized out>
The basic function "py-bt" works as expected, but inspecting Python internals
doesn't work: most local C variables are "optimized out" :-(
New attempt using a debug build:
$ LD_LIBRARY_PATH=/opt/py38debug/lib/
PYTHONPATH=/opt/py38release/lib/python3.8/site-packages gdb -args
/opt/py38debug/bin/python3.8 parse.py
... same commands to load python-gdb.py and put a breakpoint ...
Breakpoint 1, __pyx_tp_new_4lxml_5etree_iterparse (t=0x7fffea606900
<__pyx_type_4lxml_5etree_iterparse>, a=('example.xml',), k=0x0) at
src/lxml/etree.c:218930
218930 src/lxml/etree.c: No such file or directory.
(gdb) py-bt
Traceback (most recent call first):
File "parse.py", line 4, in func
context = etree.iterparse('example.xml')
File "parse.py", line 12, in <module>
func("arg")
(gdb) frame 4
#4 0x00007ffff7d6e4f8 in _PyEval_EvalFrameDefault (f=Frame 0x458790, for file
parse.py, line 4, in func (arg='arg'), throwflag=0) at Python/ceval.c:3268
3268 res = call_function(&sp, oparg, NULL);
(gdb) p f
$1 = Frame 0x458790, for file parse.py, line 4, in func (arg='arg')
(gdb) p co
$2 = (PyCodeObject *) 0x7fffea682388
(gdb) p co->co_consts
$4 = (None, 'example.xml', 'None', ' => ')
(gdb) p sp
$5 = (PyObject **) 0x458938
(gdb) p sp[0]
$6 = <module at remote 0x7fffea63a2f0>
(gdb) p sp[-1]
$7 = 'example.xml'
(gdb) p oparg
$8 = 1
The debugging experience is *much* better: it's possible to inspect Python
internals!
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue21536>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
