[issue21536] extension built with a shared python cannot be loaded with a static python

Change by STINNER Victor : -- Removed message: https://bugs.python.org/msg382882 ___ Python tracker ___ ___ Python-bugs-list

[issue21536] extension built with a shared python cannot be loaded with a static python

雅雯 added the comment: 幹你娘 -- nosy: +zhtw1234 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21536] extension built with a shared python cannot be loaded with a static python

Erik Bray added the comment: Thanks everyone. And FWIW I agree the original change is positive overall, if a bit presumptuous about different linkers' behaviors :) -- ___ Python tracker

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: > FWIW I checked the Android build after those last changes. Thank you! I was overconfident when I merged the Cygwin which looked good to me, whereas it broke Android :-( Hopefully it was quickly fixed. I hope that everything will be alright for the the

[issue21536] extension built with a shared python cannot be loaded with a static python

Xavier de Gaye added the comment: FWIW I checked the Android build after those last changes. -- ___ Python tracker ___ ___

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: New changeset b1fc41784136a2eb9737a7f0c821db52ab8d4dee by Victor Stinner (E. M. Bray) in branch 'master': bpo-21536: Fix configure.ac for LIBPYTHON on Android/Cygwin (GH-13552) https://github.com/python/cpython/commit/b1fc41784136a2eb9737a7f0c821db52ab8d4dee

[issue21536] extension built with a shared python cannot be loaded with a static python

Change by Erik Bray : -- pull_requests: +13463 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: I merged E. M. Bray's PR to get in Python 3.8 beta 1. If anything goes wrong, we can fix it later ;-) -- ___ Python tracker ___

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: New changeset c994c8fc196a167c57c8850e8abdee170d366eec by Victor Stinner (E. M. Bray) in branch 'master': bpo-21536: On Cygwin, C extensions must be linked with libpython (GH-13549)

[issue21536] extension built with a shared python cannot be loaded with a static python

Change by Erik Bray : -- pull_requests: +13461 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: > This completely breaks building extension modules on Windows-based platforms > like Cygwin and MinGW, where it is necessary when building even shared > libraries for all global symbols to resolvable at link time. C extensions are always linked to

[issue21536] extension built with a shared python cannot be loaded with a static python

Erik Bray added the comment: I vaguely recall seeing some discussion about this on python-dev or elsewhere and wish I had chimed in sooner, as I didn't realize action was going to be taken on this so soon. This completely breaks building extension modules on Windows-based platforms like

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: New changeset 4ebcd7e2980d650c711d21b93447af39a8604554 by Victor Stinner in branch 'master': bpo-21536: Update What's New in Python 3.8 entry (GH-13242) https://github.com/python/cpython/commit/4ebcd7e2980d650c711d21b93447af39a8604554 --

[issue21536] extension built with a shared python cannot be loaded with a static python

Change by STINNER Victor : -- pull_requests: +13152 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21536] extension built with a shared python cannot be loaded with a static python

Change by STINNER Victor : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue21536] extension built with a shared python cannot be loaded with a static python

reimar added the comment: As explained in Issue34814, this change not only breaks RTLD_LOCAL of libpython, but it breaks it in fact system-wide. It seems a bit much for Python to enforce a system-wide policy to not use RTLD_LOCAL just because of its own technical challenges, and thus I'd ask

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: New changeset 254b309c801f82509597e3d7d4be56885ef94c11 by Victor Stinner (xdegaye) in branch 'master': bpo-21536: On Android, C extensions are linked to libpython (GH-12989) https://github.com/python/cpython/commit/254b309c801f82509597e3d7d4be56885ef94c11

[issue21536] extension built with a shared python cannot be loaded with a static python

Xavier de Gaye added the comment: PR 12989 fixes the Android issue and has been checked on the Android emulator at API 24. Python is cross-compiled with '--enable-shared' and the python-config scripts give the expected result: generic_x86_64:/data/local/tmp/python/bin $ python -c "from

[issue21536] extension built with a shared python cannot be loaded with a static python

Change by Xavier de Gaye : -- pull_requests: +12912 stage: resolved -> patch review ___ Python tracker ___ ___ Python-bugs-list

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: I reopen the issue for the Android case. I am fine with having a different linking policy per platform. For example, Windows has a different policy than Linux. We can continue to link C extensions to libpython on Android. The ability to load release C

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: Xavier: would you be interested to work on a fix for Android? -- ___ Python tracker ___ ___

[issue21536] extension built with a shared python cannot be loaded with a static python

Xavier de Gaye added the comment: Thanks for fixing the regression Victor. Here is another potential problem. On Android API 24 built with NDK r19, symbol resolution fails in the _socket shared library after changeset 8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b when python is built with

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: New changeset 01f073f5e43329547471c846c37e23157255e30a by Victor Stinner in branch 'master': bpo-21536: Revert Makefile change on python-config (GH-12971) https://github.com/python/cpython/commit/01f073f5e43329547471c846c37e23157255e30a --

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: > Please revert the change in Makefile.pre.in made by changeset > 8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b as it breaks builds made out of the > source tree (OST). The error message is: > > make: *** No rule to make target >

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: I tested manually and I confirm that my latest change fix the compilation out of the source tree. I close again the issue. Sorry for the regression, it's now fixed. -- resolution: -> fixed status: open -> closed

[issue21536] extension built with a shared python cannot be loaded with a static python

Change by STINNER Victor : -- pull_requests: +12898 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21536] extension built with a shared python cannot be loaded with a static python

Xavier de Gaye added the comment: Please revert the change in Makefile.pre.in made by changeset 8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b as it breaks builds made out of the source tree (OST). The error message is: make: *** No rule to make target

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: Since this change is causing subtle issues like bpo-36721 "Add pkg-config python-3.8-embed", I don't think that it would be a good idea to backport this change to Python 2.7 or 3.7. I close the issue. See also bpo-36722 "In debug build, load also C

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: > If an application is linked to libpython to embed Python, the application > build requires flags to link to libpython. (...) I created bpo-36721 "Add pkg-config python-3.8-embed" to discuss this issue. -- ___

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: New changeset 8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b by Victor Stinner in branch 'master': bpo-21536: C extensions are no longer linked to libpython (GH-12946) https://github.com/python/cpython/commit/8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b --

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: If an application is linked to libpython to embed Python, the application build requires flags to link to libpython. Currently, there are: $ python3-config --libs -lpython3.7m -lcrypt -lpthread -ldl -lutil -lm $ pkg-config --libs python-3.7 -lpython3.7m

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: With an additonal change on SOABI (I will open a separated issue for that), PR 12946 allows to load lxml built in release mode in a Python built in debug mode! That's *very* useful for debugging: see my gdb example below. --- I just modified the ABI of

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: I wrote PR 12946: "On Unix, C extensions are no longer linked to libpython". Using PR 12946, the use case described in the initial message (msg218806) now works as expected. I can load a C extension built by a shared library Python with a statically linked

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: > bpo-34814 is linked to this use case: > https://bugzilla.redhat.com/show_bug.cgi?id=1585201 is an example of Python > embedded in C using dlopen("libpython2.7.so.1.0", RTLD_LOCAL | RTLD_NOW). > Problem: some C extensions of the standard library cannot be

[issue21536] extension built with a shared python cannot be loaded with a static python

Change by STINNER Victor : -- keywords: +patch pull_requests: +12870 stage: -> patch review ___ Python tracker ___ ___

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: Antoine: > Hmm, apparently the -l flag was added in #832799, for a rather complicated > case where the interpreter is linked with a library dlopened by an embedding > application (I suppose for some kind of plugin system). The OP there also > mentions

[issue21536] extension built with a shared python cannot be loaded with a static python

STINNER Victor added the comment: See also bpo-34814. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21536] extension built with a shared python cannot be loaded with a static python

Change by Antoine Pitrou : -- versions: +Python 3.7, Python 3.8 -Python 3.5 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue21536] extension built with a shared python cannot be loaded with a static python

Change by Antoine Pitrou : -- nosy: +vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21536] extension built with a shared python cannot be loaded with a static python

Changes by Piotr Dobrogost : -- nosy: +piotr.dobrogost ___ Python tracker ___

[issue21536] extension built with a shared python cannot be loaded with a static python

Changes by Antoine Pitrou pit...@free.fr: -- nosy: +lemburg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21536 ___ ___ Python-bugs-list mailing

[issue21536] extension built with a shared python cannot be loaded with a static python

Changes by Antoine Pitrou pit...@free.fr: -- nosy: +barry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21536 ___ ___ Python-bugs-list mailing

[issue21536] extension built with a shared python cannot be loaded with a static python

Antoine Pitrou added the comment: Martin, what do you think about the aforementioned use case? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21536 ___

[issue21536] extension built with a shared python cannot be loaded with a static python

Antoine Pitrou added the comment: PHP might be close to our case: Debian includes a libphp5.so (in /usr/lib/php5), yet neither /usr/bin/php5 nor the Apache libphp5.so link against it, and all the PHP modules (in /usr/lib/php5/20100525+lfs/) don't link with the shared library - on Debian.

[issue21536] extension built with a shared python cannot be loaded with a static python

Antoine Pitrou added the comment: Hmm, apparently the -l flag was added in #832799, for a rather complicated case where the interpreter is linked with a library dlopened by an embedding application (I suppose for some kind of plugin system). The OP there also mentions RTLD_GLOBAL as a

[issue21536] extension built with a shared python cannot be loaded with a static python

New submission from Antoine Pitrou: When a C extension is built (using distutils) with a shared library Python, it cannot be loaded with an otherwise identical statically linked Python. The other way round works fine. Trivial example using the _ssl module: import sys sys.path.insert(0,

[issue21536] extension built with a shared python cannot be loaded with a static python

Antoine Pitrou added the comment: Actually, it's not a -L flag but a -l flag. Removing the -lpython3.5m flag from the linker line works fine under Linux, and allows the resulting extension to be loaded with both a shared libary Python and a statically-linked Python. --

[issue21536] extension built with a shared python cannot be loaded with a static python

Martin v. Löwis added the comment: I think you are right. It would IMO be useful to research a few comparable systems. E.g. Apache modules don't link a shared library, but still refer to apr_ functions as undefined symbols - but then, there isn't an APR shared library in the first place (at