[Distutils] Re: manylinux1 guidelines for zlib?

[Antoine Pitrou]

Updating on this. It seems the manylinux1 tooling may automatically link and 
bundle an old zlib version if you link dynamically against it. That can be 
counter-productive...

Regards

Antoine.
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/UKG7EFGMHYIWU462L6BJGM2VOXLTJQ6M/

[Distutils] Re: manylinux1 guidelines for zlib?

[Tzu-ping Chung]

I see. Thanks for the clarification!


> On 05/9, 2018, at 22:54, Donald Stufft  wrote:
> 
> 
> 
>> On Sep 5, 2018, at 9:30 AM, Tzu-ping Chung > > wrote:
>> 
>> Isn’t zlib only required for compression? It is my impression that zipfile’s 
>> decompressor is pure Python,
>> and only depends on zlib if the archive is encrypted (but wheels are never 
>> encrypted).
>> 
>> zlib also does not provide decompression at its core; for that you need 
>> zlib/contrib/minizip, but I don’t
>> think CPython depends on minizip either.
>> 
>> zlib is technically optional, and not even required to installed wheels.
>> 
>> 
> 
> 
> I don’t think that’s accurate, if you look at zipfile.py, it does: 
> 
> https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f734/Lib/zipfile.py#L17-L22
>  
> 
> 
> Which attempts to import the zlib module, and if it fails sets zlib to None. 
> Then later one this is used:
> 
> https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f734/Lib/zipfile.py#L682-L683
>  
> 
> 
> Which doesn’t guard the expression at all, so I believe if you attempt to 
> unzip a wheel that uses ZIP_STORED without the zlib module built, then you’ll 
> get an AttributeError, None type does not have decompressobj exception.
> 
> 
> The Python stdlib zlib module does not appear to have any fallback at all if 
> the zlib library is not available and a recent enough version.

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/COH66FOLE5UW5HCMGGQJKLP2B34U5L6R/

[Distutils] Re: manylinux1 guidelines for zlib?

[Donald Stufft]

> On Sep 5, 2018, at 9:30 AM, Tzu-ping Chung  wrote:
> 
> Isn’t zlib only required for compression? It is my impression that zipfile’s 
> decompressor is pure Python,
> and only depends on zlib if the archive is encrypted (but wheels are never 
> encrypted).
> 
> zlib also does not provide decompression at its core; for that you need 
> zlib/contrib/minizip, but I don’t
> think CPython depends on minizip either.
> 
> zlib is technically optional, and not even required to installed wheels.
> 
> 


I don’t think that’s accurate, if you look at zipfile.py, it does: 

https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f734/Lib/zipfile.py#L17-L22
 


Which attempts to import the zlib module, and if it fails sets zlib to None. 
Then later one this is used:

https://github.com/python/cpython/blob/874809ea389e6434787e773a6054a08e0b81f734/Lib/zipfile.py#L682-L683
 


Which doesn’t guard the expression at all, so I believe if you attempt to unzip 
a wheel that uses ZIP_STORED without the zlib module built, then you’ll get an 
AttributeError, None type does not have decompressobj exception.


The Python stdlib zlib module does not appear to have any fallback at all if 
the zlib library is not available and a recent enough version.

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/5LOCEHV7BC6KNOIUXL7TLXQPIOGJZXAO/

[Distutils] Re: manylinux1 guidelines for zlib?

[Tzu-ping Chung]

Isn’t zlib only required for compression? It is my impression that zipfile’s 
decompressor is pure Python,
and only depends on zlib if the archive is encrypted (but wheels are never 
encrypted).

zlib also does not provide decompression at its core; for that you need 
zlib/contrib/minizip, but I don’t
think CPython depends on minizip either.

zlib is technically optional, and not even required to installed wheels.


> On 05/9/2018, at 19:35, Donald Stufft  wrote:
> 
> 
> 
>> On Sep 4, 2018, at 6:06 AM, Alex Walters > > wrote:
>> 
>> Since zlib is a dependency of python, the assumption has to be that it is
>> already present.
> 
> 
> It is technically an optional dependency of Python, though I don’t think you 
> can install wheels without zlib present since wheels are zip files and 
> generally include compression. Although maybe it could work in niche 
> scenarios if someone built wheels using ZIP_STORED instead of ZIP_DEFLATE?
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at 
> https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/FFB5XQBHX6N26FD5ZELMRIBRNOCNQ62Z/

--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/QA37CKADM7YOT6DZPY4IDBJ73L3ZLSXU/

[Distutils] Re: manylinux1 guidelines for zlib?

[Donald Stufft]

> On Sep 4, 2018, at 6:06 AM, Alex Walters  wrote:
> 
> Since zlib is a dependency of python, the assumption has to be that it is
> already present.


It is technically an optional dependency of Python, though I don’t think you 
can install wheels without zlib present since wheels are zip files and 
generally include compression. Although maybe it could work in niche scenarios 
if someone built wheels using ZIP_STORED instead of ZIP_DEFLATE?--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/FFB5XQBHX6N26FD5ZELMRIBRNOCNQ62Z/

[Distutils] Re: manylinux1 guidelines for zlib?

[Alex Walters]

Since zlib is a dependency of python, the assumption has to be that it is
already present.

I think you can look at the python docs for which versions of zlib you
should be dynamically linking against.  The docs state "There are known
incompatibilities between the Python module and versions of the zlib library
earlier than 1.1.3; 1.1.3 has a security vulnerability, so we recommend
using 1.1.4 or later."  So if I were linking against zlib, I would link
against 1.1.4.  However, I don't know if zlib has a stable abi... so take
that with a pinch of salt.

> -Original Message-
> From: Antoine Pitrou 
> Sent: Monday, September 3, 2018 6:51 AM
> To: distutils-sig@python.org
> Subject: [Distutils] manylinux1 guidelines for zlib?
> 
> Hello,
> 
> Surprisingly, the manylinux1 spec doesn't seem to include the zlib in the
list
> of known-to-be-available libraries (are there GNU/Linux systems out there
> without a zlib installed?).
> 
> Since I'm assuming several packages already had a need for that, is there
a
> recommended way to link in the zlib as part of a manylinux1 wheel? Would
> you recommend static linking with a private version, or dynamic linking?
> 
> Regards
> 
> Antoine.
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at https://mail.python.org/mm3/archives/list/distutils-
> s...@python.org/message/ZZG6GL3XTBLBJXSITYHEXMFKN43EREB7/
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/HLX65TU6U7N3JTUU4YRLBQNSFKZTG52X/