[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Charles Coulombe]

Charles Coulombe  added the comment:

Any update on this issue? 

This would be helpful to HPC systems that don't have libraries installed in 
standard place, and to standardize find_library as well!

--
nosy: +Charles Coulombe
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Martin Panter]

Martin Panter added the comment:

I reverted the change until we can come up with something more consistent.

--
stage: commit review -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Roundup Robot]

Roundup Robot added the comment:

New changeset 811ec2860dc4 by Martin Panter in branch 'default':
Issue #21042: Revert Linux find_library() to return just filename
https://hg.python.org/cpython/rev/811ec2860dc4

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Martin Panter]

Martin Panter added the comment:

It looks like the ldconfig parsing isn’t working for some ABIs. See the 
following buildbot failures:

* Cortex A15 armv7l: 
http://buildbot.python.org/all/builders/ARMv7%20Ubuntu%203.x/builds/3721/steps/test/logs/stdio
* ppc64le POWER8: 
http://buildbot.python.org/all/builders/PPC64LE%20Fedora%203.x/builds/769/steps/test/logs/stdio

I presume there are other flags in the ABI string, perhaps like 
(libc6,hard-float) on ARM. The code that produces these strings seems to be 
here: 
.

Looking closer at the find_library() implementation, I also realize it is not 
correct to say an absolute path is always returned. If the ldconfig check 
fails, it falls back to _get_soname(). I think we have the following options:

* Adjust the documentation to say an absolute path is only returned if the 
ldconfig call works
* Figure out how to get the right ldconfig flags for ARM and PPC
* Use the old parsing code on ARM and PPC platforms, and only return a full 
path on x86 or other platforms
* Revert the whole change

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Roundup Robot]

Roundup Robot added the comment:

New changeset 3092cf163eb4 by Martin Panter in branch 'default':
Issue #21042: Return full path in ctypes.util.find_library() on Linux
https://hg.python.org/cpython/rev/3092cf163eb4

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Martin Panter]

Martin Panter added the comment:

No I think this is ready Tamás. I have been away, but it is on my list of 
things to catch up on. I won’t add any What’s New entry.

--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Tamás Bence Gedai]

Tamás Bence Gedai added the comment:

Is there anything else that I can do for this issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Berker Peksag]

Berker Peksag added the comment:

I don't think we need an entry in whatsnew/3.6.rst for this (we already have 
documented the change in a versionchanged directive and a Misc/NEWS item)

--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Martin Panter]

Martin Panter added the comment:

Thanks, this looks pretty good to me. I just need to remember to write a What’s 
New entry.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Tamás Bence Gedai]

Tamás Bence Gedai added the comment:

Updated the patch to remove the code duplication, now it stores the values that 
are calculated in the setUpClass method. It was a good and simple idea, I 
should have come up with it... :)

I'm pretty sure I got the errors during configuration because of the gcc 
version I have.

--
Added file: http://bugs.python.org/file42003/find_lib_v4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Martin Panter]

Martin Panter added the comment:

I left a suggestion about the duplication in the code review.

I set CC with “configure” like you said. I also had to run “make clean” to get 
rid of the old 64-bit stuff. But it might depend on the GCC that you have 
installed. On Arch Linux, I have gcc-multilib, which supports 64-bit and 32-bit 
targets.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Tamás Bence Gedai]

Tamás Bence Gedai added the comment:

I've added a new method to Test_OpenGL_libs as you suggested. I check whether 
find_library returns an absolute path. Note that I didn't distinguish different 
systems, as according to the docs, only Linux systems return the file name, 
other systems return the absolute path. 
(https://docs.python.org/3.5/library/ctypes.html#ctypes-reference) An other 
thing to note, that I introduced some code duplication as I use the same code 
snippet from setUpClass method to figure out the correct parameters to 
find_library.

The patch uses the same regex as you gave.

By the way, what do I have to do to compile CPython on a 64 bit system in 32 
bit mode? I tried ./configure CC="gcc -m32" but it gave me an error. Is it the 
correct way?

Also, I signed contributor agreement.

--
Added file: http://bugs.python.org/file41992/find_lib_v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Martin Panter]

Martin Panter added the comment:

Tamás, it might be a good idea for you to sign a contributor agreement 
.

I compiled Python in 32-bit mode and tried your v2 patch out, which found the 
wrong library as I predicted. Then I tried your new regex and it picked out the 
correct line. I had to edit it to get it to extract just the filename from the 
line:

r'lib%s\.[^\s]+\s\(%s(?:,\s.*)?\)\s=>\s(.*)' % (re.escape(name), abi_type)

I factored out the closing bracket from the comma + space bit, moved the group 
brackets to the end to extract the filename, and made it a raw string.

Without the patch, in 32-bit mode it will find 64-bit-only libraries:

>>> find_library("m")  # 32- and 64-bit available
'libm.so.6'
>>> find_library("tcl8.6")  # Only 64-bit version available!
'libtcl8.6.so'

With my edited regex:

>>> find_library("m")
'/usr/lib32/libm.so.6'
>>> find_library("tcl8.6") is None  # No 32-bit version found
True

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Tamás Bence Gedai]

Tamás Bence Gedai added the comment:

What do you think about this regex?

'(lib%s\.[^\s]+\s\(%s(?:\)|,\s.*\))\s=>\s.*)' % (re.escape(name), abi_type))

It works on 64 bit, just like before, but I could not test it on 32 bit. I'll 
add tests soon.

I looked for documentation on ldconfig, but could not find anything useful.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Martin Panter]

Martin Panter added the comment:

I think the new regular expression will still find the wrong library in my libm 
example above. In 32-bit mode, it will be only looking to match \(libc6.*\). 
Since my example has the 64-bit line first, that one will match first. (I 
haven’t actually tested this, but I think I compiled 32-bit Python once before 
just by specifying CC="gcc -m32". Sorry to keep poking holes in your regular 
expression :)

Do you know if there is documentation for the “ldconfig -p” output format, or 
do we just have to go on what we see? If so, I would change it to ensure the 
ABI type string is either followed by a comma and space ", " or a closing 
bracket ")". A comma on its own, or other letters, is not a match.

I did a search for “find_library”, and the most likely place is 
/Lib/ctypes/test/test_find.py. You could probably get away with just adding a 
new method like Test_OpenGL_libs.test_path(). On my computer I have the GL and 
GLU libraries (but not gle), so I guess that these libraries are fairly common 
(plus it already has Windows and OS X versions to test).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Tamás Bence Gedai]

Tamás Bence Gedai added the comment:

I fixed the ABI matching, it was a stupid mistake, thanks for pointing it out 
:) I think now it works as expected.

I really don't find a place for testing. Maybe a new test file could be added, 
but I think the testing code for find_library wouldn't be more than 10 lines. 
Do you have any suggestions?

Thanks Martin for all your patience :)

--
Added file: http://bugs.python.org/file41866/find_lib_v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Martin Panter]

Martin Panter added the comment:

The ABI matching looks wrong to me. If I am looking for a 32-bit library, won’t 
it incorrectly catch the wrong path in the following “ldconfig -p” output:

'\tlibm.so.6 (libc6,x86-64, OS ABI: Linux 2.6.32) => /usr/lib/libm.so.6\n'
'\tlibm.so.6 (libc6, OS ABI: Linux 2.6.32) => /usr/lib32/libm.so.6\n'

Perhaps the abi_type check needs to be moved in front of the path name 
extraction.

For the version, I would put 3.6. Since this changes documented behaviour and 
has the potential to break compatibilty, it is best not to change it in a bug 
fix release. (3.5 has already been released.)

The problem with the test() function in ctypes.util is that it is not run by 
the main Python regression test suite. The tests under ctypes/test/ are run by 
the test suite.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Tamás Bence Gedai]

Tamás Bence Gedai added the comment:

Added a new patch, as Martin pointed out, I put back the ABI matching. The 
regex looks quite ugly, because it has to match \n\t. To be exact, it has to 
match something like this: "/lib/x86_64-linux-gnu/libc.so.6\n\tlibbz2.so.1.0 
(libc6,x86-64)".

I updated the docs, although I don't know what should I write for the version, 
please someone help me with that.

For testing, there's a test function in this module, I updated that.

--
Added file: http://bugs.python.org/file41835/find_lib_v1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Gedai Tamás Bence]

Gedai Tamás Bence added the comment:

Added a small patch that solves this issue on Ubuntu 15.10.

Produces output like:
/lib/x86_64-linux-gnu/libm.so.6
/lib/x86_64-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/libbz2.so.1.0

I'd be glad to add some test cases if someone can give me some tips on how to 
do that.

--
keywords: +patch
nosy: +beng94
Added file: http://bugs.python.org/file41789/find_lib.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Martin Panter]

Martin Panter added the comment:

IMO this should be treated as a new feature for the next release. But 
consistently returning the path sounds good to me if there is no good reason 
not to.

Left a question on the review. I think you also need to update the 
documentation, and since this is changing documented behaviour it probably 
needs a What’s New entry.

For tests, I would try doing something like find_library("c"), and ensuring 
that the result is an absolute path. We may have to end up skipping the test 
for platforms like Windows where this is not expected to work. Look through the 
files in /Lib/ctypes/test/ for a good place for it to live (if there isn’t 
already a test there you can modify).

--
nosy: +martin.panter
stage:  -> patch review
type:  -> enhancement
versions: +Python 3.6 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21042] ctypes.util.find_library() should return full pathname instead of filename in linux

[Hernan Grecco]

New submission from Hernan Grecco:

In Windows and OSX, `find_library` returns the full pathname of the library 
file. But on Linux, it returns just the filename. Is there a reason for this 
difference? 

For consistency, it would be better to return the full pathname in all cases. 
It is easy to get the filename from the full pathname, but not the other way 
around.

--
components: ctypes
messages: 214647
nosy: Hernan.Grecco
priority: normal
severity: normal
status: open
title: ctypes.util.find_library() should return full pathname instead of 
filename in linux
versions: Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21042
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com