[issue13508] ctypes' find_library breaks with ARM ABIs

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Trevor Newton]

Trevor Newton  added the comment:

I am still encountering this issue when using pyudev on Python 3.8.5.

--
nosy: +trevor.newton

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 2.7, Python 3.2, Python 
3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Yegor Yefremov]

Yegor Yefremov added the comment:

This issue is still up to date. Most impacted are distos like for example 
Buldroot (http://buildroot.org/). Usual production rootfs provide neither gcc 
nor /sbin/ldconfig nor objdump. So find_library() is doomed from the very 
beginning.

Such packages like pyusb, pyudev etc. rely on find_library() and are not usable 
in such environment. Pyusb developers had to create an extra backend to 
overcome this issue (https://github.com/walac/pyusb/pull/29).

So common solution would be very useful for such environments.

--
nosy: +Yegor Yefremov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
nosy: +belopolsky

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
nosy: +eric.araujo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +haypo
stage: needs patch - patch review
versions: +Python 3.3 -Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Stefano Rivera]

Changes by Stefano Rivera pyt...@rivera.za.net:


--
nosy: +stefanor

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Barry A. Warsaw]

Changes by Barry A. Warsaw ba...@python.org:


--
nosy: +barry

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Loïc Minier]

New submission from Loïc Minier l...@dooz.org:

Hi,

This bug was originally reported at 
https://bugs.launchpad.net/bugs/898172

ctypes/utils.py provides a find_library function which amongst other things 
will scan the ldconfig -p output on linux to find libraries by name.  It 
applies some logic to filter out incompatible libraries, however the logic is 
mainly based on uname output which is incorrect.

We noticed because the new Debian/Ubuntu armhf ports have a slightly different 
ldconfig -p output than the armel ports; one gets ,hard-float in the output, 
e.g.:
ld-linux.so.3 (libc6,hard-float) = 
/lib/arm-linux-gnueabihf/ld-linux.so.3

there's provision in find_library to allow for certain strings when uname 
returns certain names:
 mach_map = {
 'x86_64-64': 'libc6,x86-64',
 'ppc64-64': 'libc6,64bit',
 'sparc64-64': 'libc6,64bit',
 's390x-64': 'libc6,64bit',
 'ia64-64': 'libc6,IA-64',

but this is incorrect for multiple reasons:
a) this requires setting utsname properly before running a 32-bits python on a 
64-bits kernel (e.g. linux32 ./foo.py instead of just ./foo.py); this 
shouldn't be needed and breaks 32-bits userspace installations with a 64-bits 
kernel
b) uname output can be anything really, e.g. i486, i586, i686 etc. on 32-bits 
x86, or armv5l, armv6l, armv7l etc. on ARM
c) uname output doesn't indicate userspace ABI, a single kernel can support 
multiple ABIs; for instance ARM kernels can support EABI and OABI (old ABI) 
syscall ABIs at the same time, and even with the same syscall ABI like EABI the 
userspace calling conventions might allow for multiple ABIs to be present on 
the filesystem -- for instance soft-float and hard-float userspace calling 
conventions

I've attached a patch to ctypes/utils.py in the Launchpad bug which I'll also 
attach here.  It will work for either soft-float or hard-float, but not if 
ldconfig -p lists both types of libraries (as will be the case with biarch or 
multiarch systems).

It is extremely hard to reproduce correct glibc semantics in find_library, and 
a linux implementation would necessarily become extremely glibc and linux 
specific.  One possible way is to look at /proc/$pid/maps output to find 
information about the ABI of the currently running program, and then ask the 
runtime linker (ld.so) to check whether a given library is compatible or not 
(--verify).  Another way would be to run ldd on sys.executable to find the 
runtime linker or libc.  This is all extremely fragile and linux andglibc 
specific, and will likely fail in special cases.

Finally, one needs to wonder whether offering find_library as an API isn't 
calling for trouble; dlopen() requires one to state which SOVER should be used, 
e.g. dlopen(libmagic.so.1), not dlopen(magic).  Allowing the first SOVER to 
be used means that the behavior is not determinstic and also means that people 
wont think of binary compatibility when implementing ctypes-based bindings.  I 
would personally prefer if this API was deprecated and if we recommended for 
upstreams to use ctypes.cdll.LoadLibrary(libmagic.so.1) constructs.

Cheers,

--
components: ctypes
messages: 148656
nosy: lool
priority: normal
severity: normal
status: open
title: ctypes' find_library breaks with ARM ABIs
type: behavior
versions: Python 2.6, Python 2.7, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Meador Inge]

Changes by Meador Inge mead...@gmail.com:


--
nosy: +meador.inge
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Loïc Minier]

Changes by Loïc Minier l...@dooz.org:


--
keywords: +patch
Added file: http://bugs.python.org/file23817/ctypes-arm.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13508] ctypes' find_library breaks with ARM ABIs

[Loïc Minier]

Loïc Minier l...@dooz.org added the comment:

While I'm at it, find_library also tries creating temp files when running gcc 
and other issues mention trouble running gcc or propose running ld:
http://bugs.python.org/issue9998
http://bugs.python.org/issue5289

IMHO, calling binutils/gcc is troublesome, it's not necessarily installed on 
production systems and creating tempfiles when running a program just to locate 
a library is fragile at best.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com