Bug#736721: Incorrect dependencies
On 2014-02-07 15:33 , Jonas Borgström wrote:
On 2014-02-07 15:08 , Julien Cristau wrote:
On Fri, Feb 7, 2014 at 15:04:47 +0100, Jonas Borgström wrote:
On 2014-02-07 12:12 , Jakub Wilk wrote:
What attic/crypto.py currently does is:
libcrypto = cdll.LoadLibrary(find_library('crypto'))
But there is no guarantee that find_library('crypto') returns a library
that is ABI-compatible with the Python code. For the Debian package, a
quickdirty fix is to replace the find_library() call with
'libcrypto.so.1.0.0', then hardcode libssl1.0.0 in Depends. But that of
course means that the package would need a sourceful upload by the next
OpenSSL transition.
You're right. The find_library('crypto') line itself won't guarantee
any ABI-compatibility but a Depends: libssl1.0.0 (= 1.0.0) line will.
No, because it doesn't guarantee that libssl1.1 is not installed, and
find_library('crypto') may well pick that one instead of the expected
libcrypto.so.1.0.0.
Ok, I see your point. How about this then:
- Use libssl1.0.0 = 1.0.0 as a dependency
- I change the code to only fall back on find_library('crypto') if
libcrypto.so.1.0.0 is not found in the usual place?
That way we're ready when/if libssl 1.1 is released _and_ ships with a
different ABI for the 4 libcrypto function used by Attic.
All of this will hopefully be a non-issue in a not so distant future
since I'm thinking about to replace all python extension and ctypes code
with cffi in order to support PyPy.
Hi all,
After a failed attempt at replacing all cython and ctypes usage with
cffi (cffi is about 40% slower than cython for some operations with
cPython), I decided to instead replace ctypes with cython.
So as of this commit:
https://github.com/jborg/attic/commit/0e39acffd3c0dc74738e7acf77994d8a6bffa56e
The attic.crypto module is now a regular python extension module that
is properly linked with libcrypto.
This change will be part of Attic 0.12 once released but it also cleanly
applies to the newly released Attic 0.11 if anyone is interested.
/ Jonas
signature.asc
Description: OpenPGP digital signature
Bug#736721: Incorrect dependencies
On 2014-02-07 08:37 , Dmitry Shachnev wrote: Hi Clint, See http://docs.python.org/3/library/ctypes.html for description of how ctypes works. As Python programs are interpreted, the only way to access external shared libraries from Python code is by dlopen()ing them. By looking at attic code, it only accesses libc (in xattr.py) and libcrypto (in attic/crypto.py). Of course, this is not the best practice, a simplier solution will be to use existing modules like python-xattr and python-crypto, or writing a Python extension where you can use C code. The original reasons why Attic doesn't use python-crypto and python-xattr is that they did not have any working python3 support at the time. Attic also uses PBKDF2 to passphrase-protect key files and that functionality has only recently been added to python-crypto. And even now it is 11 times slower than the libcrypto version which translates into Attic having to use 11 times fewer iterations which will make the passphrase 11 times easier to brute force. So I rather not use python-crypto until this improves. python-xattr seems to have improved a lot since I last looked at it so we might be able to switch to that if their Darwin and FreeBSD support works as intended. So if this indeed is a showstopper I'll look into replacing the ctypes-bits with python extensions or cffi. / Jonas signature.asc Description: OpenPGP digital signature
Bug#736721: Incorrect dependencies
What attic/crypto.py currently does is:
libcrypto = cdll.LoadLibrary(find_library('crypto'))
But there is no guarantee that find_library('crypto') returns a library
that is ABI-compatible with the Python code. For the Debian package, a
quickdirty fix is to replace the find_library() call with
'libcrypto.so.1.0.0', then hardcode libssl1.0.0 in Depends. But that of
course means that the package would need a sourceful upload by the next
OpenSSL transition.
* Dmitry Shachnev mity...@gmail.com, 2014-02-07, 11:37:
a simplier solution will be to use existing modules like python-xattr
and python-crypto, or writing a Python extension where you can use C
code.
Instead of full-blow extension module, you could build a thin C wrapper
over the interesting libcrypto functions, and then dlopen the wrapper
instead of the shared library.
But then I notice that attic already contains some extensions modules
written in Cython, so surely rewritting crypto.py and xattr.py in Cython
shouldn't be a big deal? :-)
--
Jakub Wilk
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#736721: Incorrect dependencies
On 2014-02-07 12:12 , Jakub Wilk wrote:
What attic/crypto.py currently does is:
libcrypto = cdll.LoadLibrary(find_library('crypto'))
But there is no guarantee that find_library('crypto') returns a library
that is ABI-compatible with the Python code. For the Debian package, a
quickdirty fix is to replace the find_library() call with
'libcrypto.so.1.0.0', then hardcode libssl1.0.0 in Depends. But that of
course means that the package would need a sourceful upload by the next
OpenSSL transition.
You're right. The find_library('crypto') line itself won't guarantee
any ABI-compatibility but a Depends: libssl1.0.0 (= 1.0.0) line will.
This thread started because I noticed that the package unnecessarily
listed python3-openssl as a dependency (and python3-openssl depends on
libssl1.0.0 (= 1.0.0)).
So I suggested to drop the middleman and replace the Depends:
python3-openssl line with Depends: libssl1.0.0 (= 1.0.0) since the
python3-openssl code itself isn't needed, just the libssl1.0.0 dependency.
So from my point of view this somehow changed from being about a one
line change/improvement in debian/control to a discussion about python
best practices and code rewrite.
That said, Debian packages are important and I'm willing to do whatever
changes it takes to make Attic debian compatible.
/ Jonas
signature.asc
Description: OpenPGP digital signature
Bug#736721: Incorrect dependencies
On 2014-02-07 15:08 , Julien Cristau wrote:
On Fri, Feb 7, 2014 at 15:04:47 +0100, Jonas Borgström wrote:
On 2014-02-07 12:12 , Jakub Wilk wrote:
What attic/crypto.py currently does is:
libcrypto = cdll.LoadLibrary(find_library('crypto'))
But there is no guarantee that find_library('crypto') returns a library
that is ABI-compatible with the Python code. For the Debian package, a
quickdirty fix is to replace the find_library() call with
'libcrypto.so.1.0.0', then hardcode libssl1.0.0 in Depends. But that of
course means that the package would need a sourceful upload by the next
OpenSSL transition.
You're right. The find_library('crypto') line itself won't guarantee
any ABI-compatibility but a Depends: libssl1.0.0 (= 1.0.0) line will.
No, because it doesn't guarantee that libssl1.1 is not installed, and
find_library('crypto') may well pick that one instead of the expected
libcrypto.so.1.0.0.
Ok, I see your point. How about this then:
- Use libssl1.0.0 = 1.0.0 as a dependency
- I change the code to only fall back on find_library('crypto') if
libcrypto.so.1.0.0 is not found in the usual place?
That way we're ready when/if libssl 1.1 is released _and_ ships with a
different ABI for the 4 libcrypto function used by Attic.
All of this will hopefully be a non-issue in a not so distant future
since I'm thinking about to replace all python extension and ctypes code
with cffi in order to support PyPy.
/ Jonas
signature.asc
Description: OpenPGP digital signature
Bug#736721: Incorrect dependencies
On Fri, Feb 7, 2014 at 15:04:47 +0100, Jonas Borgström wrote:
On 2014-02-07 12:12 , Jakub Wilk wrote:
What attic/crypto.py currently does is:
libcrypto = cdll.LoadLibrary(find_library('crypto'))
But there is no guarantee that find_library('crypto') returns a library
that is ABI-compatible with the Python code. For the Debian package, a
quickdirty fix is to replace the find_library() call with
'libcrypto.so.1.0.0', then hardcode libssl1.0.0 in Depends. But that of
course means that the package would need a sourceful upload by the next
OpenSSL transition.
You're right. The find_library('crypto') line itself won't guarantee
any ABI-compatibility but a Depends: libssl1.0.0 (= 1.0.0) line will.
No, because it doesn't guarantee that libssl1.1 is not installed, and
find_library('crypto') may well pick that one instead of the expected
libcrypto.so.1.0.0.
Cheers,
Julien
--
Julien Cristau julien.cris...@logilab.fr
Logilab http://www.logilab.fr/
Informatique scientifique gestion de connaissances
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#736721: Incorrect dependencies
On Sun, Jan 26, 2014 at 01:04:09PM +0100, Jonas Borgström wrote: Attic uses ctypes to interact with libcrypto and does not use python3-openssl. So the python3-openssl dependency should be replaced with a dependency on libssl1.0.0. I assume this means this cdll thing is calling dlopen() on libcrypto. Maybe one of the Python folks can tell me how to detect that and correlate it with shlibs in an automated fashion, or better yet adopt the package. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#736721: Incorrect dependencies
Hi Clint, See http://docs.python.org/3/library/ctypes.html for description of how ctypes works. As Python programs are interpreted, the only way to access external shared libraries from Python code is by dlopen()ing them. By looking at attic code, it only accesses libc (in xattr.py) and libcrypto (in attic/crypto.py). Of course, this is not the best practice, a simplier solution will be to use existing modules like python-xattr and python-crypto, or writing a Python extension where you can use C code. -- Dmitry Shachnev On Thu, Feb 6, 2014 at 9:00 PM, Clint Adams cl...@softwarefreedom.org wrote: On Sun, Jan 26, 2014 at 01:04:09PM +0100, Jonas Borgström wrote: Attic uses ctypes to interact with libcrypto and does not use python3-openssl. So the python3-openssl dependency should be replaced with a dependency on libssl1.0.0. I assume this means this cdll thing is calling dlopen() on libcrypto. Maybe one of the Python folks can tell me how to detect that and correlate it with shlibs in an automated fashion, or better yet adopt the package. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#736721: Incorrect dependencies
Package: attic Version: 0.8.1-1 Attic uses ctypes to interact with libcrypto and does not use python3-openssl. So the python3-openssl dependency should be replaced with a dependency on libssl1.0.0. Thanks for packaging Attic btw! I've added some info about that to the home page. Attic 0.9 is also out now. / Jonas signature.asc Description: OpenPGP digital signature
