[issue16754] Incorrect shared library extension on linux
Éric Araujo added the comment:
Amaury’s questions are still unanswered:
So sysconfig.get_config_var('SO') will change in a micro release?
Won't this break working user code? Give unexpected file names?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16754
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Arfrever Frehtes Taifersar Arahesis added the comment: There were already 5 releases of Python 3.3 (starting with 3.3.1) with this change included. The number of affected applications was rather small. I had seen only NumPy. -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Roundup Robot added the comment: New changeset ccb679e5ae0e by Éric Araujo in branch 'default': Update SOABI PEP to reflect config var change (#16754). http://hg.python.org/peps/rev/ccb679e5ae0e -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Changes by Barry A. Warsaw ba...@python.org: -- assignee: barry - doko ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Éric Araujo added the comment: This change may be responsible for #17869 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Éric Araujo added the comment: False alarm, disregard my previous message. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Changes by kent fuzzba...@comcast.net: -- nosy: +Thekent ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Julian Taylor added the comment: is SHLIB_SUFFIX=.so really correct on mac? shared libraries have .dylib extension, loadable modules have .so (which would be EXT_SUFFIX?) e.g libpython itself uses .dylib. -- nosy: +jtaylor108 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Matthias Klose added the comment: are you speculating, or is your comment based on some testing? MacOS had this value before, and apparently it did work before that change. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Julian Taylor added the comment: I'm going by what says in configure: # SHLIB_SUFFIX is the extension of shared libraries The extension of shared libraries on macos is .dylib in most cases (e.g libtool based libraries and as mentioned python itself) Maybe its just a documentation/naming issue. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Julian Taylor added the comment: just to clarify its not any issue in python, python is working fine with .so The issue is just that theses variables tends to be used by other applications to figure out information on the system (like shared library extension, see numpy.distutils) You certainly could argue that these applications are broken by even needing this information, but a proper naming of the variables could help reduce confusion and wrong code. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
A.M. Kuchling added the comment: I just tried building trunk on MacOS 10.6.8, and make install fails with this traceback: Traceback (most recent call last): File ./setup.py, line 2175, in module main() File ./setup.py, line 2170, in main Tools/scripts/2to3, Tools/scripts/pyvenv] File /Users/akuchling/source/p/cpython/Lib/distutils/core.py, line 148, in setup dist.run_commands() File /Users/akuchling/source/p/cpython/Lib/distutils/dist.py, line 917, in run_commands self.run_command(cmd) File /Users/akuchling/source/p/cpython/Lib/distutils/dist.py, line 936, in run_command cmd_obj.run() File /Users/akuchling/source/p/cpython/Lib/distutils/command/install.py, line 566, in run self.run_command(cmd_name) File /Users/akuchling/source/p/cpython/Lib/distutils/cmd.py, line 313, in run_command self.distribution.run_command(command) File /Users/akuchling/source/p/cpython/Lib/distutils/dist.py, line 936, in run_command cmd_obj.run() File /Users/akuchling/source/p/cpython/Lib/distutils/command/install_lib.py, line 93, in run outfiles = self.install() File ./setup.py, line 2068, in install self.set_file_modes(outfiles, 0o644, 0o755) File ./setup.py, line 2079, in set_file_modes if filename.endswith(self.shlib_suffix): mode = sharedLibMode TypeError: endswith first arg must be str or a tuple of str, not NoneType make: *** [sharedinstall] Error 1 The following patch fixes the traceback, but I don't know why shlib_suffix is None. - hg diff diff -r 9328e2b8a397 setup.py --- a/setup.py Tue Apr 02 22:13:49 2013 +0200 +++ b/setup.py Tue Apr 02 19:10:27 2013 -0400 @@ -2076,7 +2076,9 @@ for filename in files: if os.path.islink(filename): continue mode = defaultMode -if filename.endswith(self.shlib_suffix): mode = sharedLibMode +if (self.shlib_suffix is not None and +filename.endswith(self.shlib_suffix)): +mode = sharedLibMode log.info(changing mode of %s to %o, filename, mode) if not self.dry_run: os.chmod(filename, mode) -- nosy: +akuchling ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Ned Deily added the comment: Did you start with a clean build directory and rerun ./configure? What ./configure options did you use? Make sure that an existing install isn't getting in the way during a build, particularly with --enable-shared. With a current tip of the default branch, I've built all three standard OS X configurations (default unix (non-shared), --enable-shared, and --enable-framework) on OS X 10.8 and make install succeeds for all of them with sysconfig.get_config_var(SHLIB_SUFFIX) returning .so as expecting. -- nosy: +ned.deily ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Amaury Forgeot d'Arc added the comment:
So sysconfig.get_config_var('SO') will change in a micro release?
Won't this break working user code? Give unexpected file names?
--
nosy: +amaury.forgeotdarc
status: pending - open
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16754
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Roundup Robot added the comment: New changeset 8f91014c8f00 by doko in branch '3.2': - Issue #16754: Fix the incorrect shared library extension on linux. Introduce http://hg.python.org/cpython/rev/8f91014c8f00 New changeset 14f0c9e595a5 by doko in branch '3.3': - Issue #16754: Fix the incorrect shared library extension on linux. Introduce http://hg.python.org/cpython/rev/14f0c9e595a5 New changeset f6a6b4eed5b0 by doko in branch 'default': - Issue #16754: Fix the incorrect shared library extension on linux. Introduce http://hg.python.org/cpython/rev/f6a6b4eed5b0 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Matthias Klose added the comment: fixed, pep update is pending -- assignee: eric.araujo - barry resolution: - fixed stage: - committed/rejected status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Sean Reifschneider added the comment: Toshio and Matthias: This approach seems sane to me, Nick asked me to review this ticket. I'm not coming up with any objections. +1 for retiring SO at some point after 3.2, and EXT_SUFFIX and SHLIB_SUFFIX. What documentation needs to be changed? PEP 3149 (maybe just reference this being obsolete)? What other documentation? -- nosy: +jafo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Matthias Klose added the comment: proposed PEP change. Or should the change itself be documented in the PEP? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Changes by Matthias Klose d...@debian.org: Added file: http://bugs.python.org/file29514/pep.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Toshio Kuratomi added the comment: Matthias, Barry, and I looked at this at pycon today. It looks a bit like the original intent was to have SO = .so SOABI = cpython-32mu and then CPython extension module suffixes would be: if SOABI: so_ext = ''.join(., SOABI, SO) else: so_ext = SO This would need to be used in distutils/commands/build_ext.py We weren't sure if there are other places in the code that would need it as well but a quick build of a module which uses libraries and needs C extensions showed that this seems to work. The one worrisome question is whether more people have come to rely on the SO variable holding the extension module suffix or if more code was broken by the extension module suffix replacing the library suffix in the SO variable. Answering that might better show us whether to change these variables back to their original meanings or to create brand new variables that have the correct values. We also discovered the reason the current version appears to work with python-pillow on Ubuntu boxes but not Fedora. The find_library_files() code first checks for library names that would match with the shared library suffix. If that fails, it falls back to looking for the static library suffix. On Fedora, there are no static libraries so this function just fails to find the library. On Ubuntu, the code finds the static libraries and returns that. This causes the code in python-pillow to attempt to link to the library with -ljpeg, -lpng, etc... Since the shared libraries actually are present, the compiler and linker use the shared versions even though python only found the static versions. -- nosy: +a.badger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Changes by Nick Coghlan ncogh...@gmail.com: -- nosy: +ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Matthias Klose added the comment: the patch so.diff - introduces two new macros EXT_SUFFIX and SHLIB_SUFFIX, and uses these instead of the SO macro. - keeps the SO macro, and restores it to it's original value - Removes the SHLIB_EXT preprocessor define (only used in Python/dynload_hpux.c, and there we know that we will end up with .sl anyway. For trunk maybe consider removing the SO macro, I think it's better to change it back to it's original value for 3.1 and 3.2, knowing that it will probably break some Debian packaging, but that can be fixed too. -- keywords: +patch nosy: +doko Added file: http://bugs.python.org/file29494/so.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Changes by Bohuslav Slavek Kabrda bkab...@redhat.com: -- nosy: +bkabrda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Roumen Petrov added the comment: This is issue introduced with implementation of SOABI. Build of standard extensions is protected by following code: - class PyBuildExt(build_ext): def __init__(self, dist): build_ext.__init__(self, dist) self.failed = [] def build_extensions(self): # Detect which modules should be compiled old_so = self.compiler.shared_lib_extension # Workaround PEP 3149 stuff self.compiler.shared_lib_extension = os.environ.get(SO, .so) try: missing = self.detect_modules() finally: self.compiler.shared_lib_extension = old_so -- I think that PEP 3149 is not accurate . For historical reasons (backward compatibility) SO must remain same as OS specific suffix and and new variable is required for python specific suffix. -- nosy: +rpetrov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Dave Malcolm added the comment:
For reference, quoting PEP 3149:
sysconfig.get_config_var('SO')
'.cpython-32mu.so'
--
nosy: +dmalcolm
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16754
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Sandro Mani added the comment: So, from what I can see, historically the SO extension was taken from sysconfig.py, see [1] lines 24 and 60. Then, the CCompiler class got overhauled, and the value was hardcoded to .so, see [2], but the compiler.shared_lib_extension = SO statement remained, presumably oversight. So technically, this is also affecting python 2.x, though in that case it does not make any difference, since python2 does not use versioned so-names for binary modules. [1] http://hg.python.org/cpython/file/2802fb52e99b/Lib/distutils/unixccompiler.py [2] http://hg.python.org/cpython/rev/7922d08426ca -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
Changes by Éric Araujo mer...@netwok.org: -- nosy: +barry versions: +Python 3.2, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16754 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16754] Incorrect shared library extension on linux
New submission from Sandro Mani:
I'm using Python3 as available in Fedora rawhide
(python3-3.3.0-2.fc19.x86_64).
Attempting to build a project using python3/distutils, I noticed that
find_library_file would not find any library at all. Some investigation
showed that this was due to the fact that libraries were searched with
the .cpython-33m.so extension. Even more investigation showed that the
library extension was read being overridden by the one defined in the
/usr/lib64/python3.3/config-3.3m/Makefile shipped by python3-libs. See
below for the detailed analysis. The python-versioned extension
obviously makes no sense for regular shared objects which are not python
binary modules, so this is clearly wrong. As a workaround I patched
sysconfig.py to comment out customize_compiler::235
(compiler.shared_lib_extension =
so_ext, see below), recompiled python (all tests still pass), and things seem
to work.
Detailed analysis:
setup.py:
def _find_library_file(self, library):
return self.compiler.find_library_file(self.compiler.library_dirs,
library)
---
In function
/usr/lib64/python3.3/distutils/unixcompiler.py at find_library_file::266:
shared_f = self.library_filename(lib, lib_type='shared')
In function
/usr/lib64/python3.3/distutils/ccompiler.py at library_filename::882:
ext = getattr(self, lib_type + _lib_extension)
- Where does shared_lib_extension get defined?
* At /usr/lib64/python3.3/distutils/ccompiler.py::66
shared_lib_extension = None - default for abstract class
* At /usr/lib64/python3.3/distutils/unixcompiler.py::77
shared_lib_extension = .so - this is the correct value
* In function
/usr/lib64/python3.3/distutils/sysconfig.py at customize_compiler::235
by /usr/lib64/python3.3/distutils/sysconfig.py at customize_compiler::235
compiler.shared_lib_extension = so_ext
by /usr/lib64/python3.3/distutils/sysconfig.py at customize_compiler::194
(cc, cxx, opt, cflags, ccshared, ldshared, so_ext, ar, ar_flags) = \
get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',
'CCSHARED', 'LDSHARED', 'SO', 'AR', 'ARFLAGS'))
by /usr/lib64/python3.3/distutils/sysconfig.py at get_config_vars::530
526 global _config_vars
527 if _config_vars is None:
528 func = globals().get(_init_ + os.name) # - os.name = posix
529 if func:
530 func() # - _init_posix, populates _config_vars
by /usr/lib64/python3.3/distutils/sysconfig.py at _init_posix::439
435 g = {}
436 # load the installed Makefile:
437 try:
438 filename = get_makefile_filename() #
/usr/lib64/python3.3/config-3.3m/Makefile
439 parse_makefile(filename, g)
...
476 global _config_vars
477 _config_vars = g # - _config_vars[SO] = .cpython-33m.so
by /usr/lib64/python3.3/config-3.3m/Makefile::122
SO= .cpython-33m.so
--
assignee: eric.araujo
components: Distutils
messages: 177979
nosy: eric.araujo, smani, tarek
priority: normal
severity: normal
status: open
title: Incorrect shared library extension on linux
type: behavior
versions: Python 3.3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16754
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
