[issue40698] distutils.command.upload md5_digest

[Christian Heimes]

Christian Heimes  added the comment:

Thanks Charis and Donald!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40698] distutils.command.upload md5_digest

[miss-islington]

miss-islington  added the comment:


New changeset f541a371a5e608517314a106012e0c19739d2d02 by Miss Islington (bot) 
in branch '3.9':
bpo-40698: Improve distutils upload hash digests (GH-20260)
https://github.com/python/cpython/commit/f541a371a5e608517314a106012e0c19739d2d02


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40698] distutils.command.upload md5_digest

[miss-islington]

miss-islington  added the comment:


New changeset e572c7f6dbe5397153803eab256e4a4ca3384f80 by Christian Heimes in 
branch 'master':
bpo-40698: Improve distutils upload hash digests (GH-20260)
https://github.com/python/cpython/commit/e572c7f6dbe5397153803eab256e4a4ca3384f80


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40698] distutils.command.upload md5_digest

[miss-islington]

Change by miss-islington :


--
pull_requests: +19547
pull_request: https://github.com/python/cpython/pull/20261

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40698] distutils.command.upload md5_digest

[Christian Heimes]

Christian Heimes  added the comment:

Thanks for your elaborate explanation, Donald!

I have implemented your proposal in PR 20260.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40698] distutils.command.upload md5_digest

[Christian Heimes]

Change by Christian Heimes :


--
keywords: +patch
pull_requests: +19546
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/20260

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40698] distutils.command.upload md5_digest

[Christian Heimes]

Christian Heimes  added the comment:

Charis pointed me to https://github.com/pypa/warehouse/issues/681 / 
https://github.com/pypa/warehouse/pull/891

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40698] distutils.command.upload md5_digest

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

There is also https://github.com/pypa/warehouse/pull/888

So I would assume it's safe it change the digest to sha256.

--
nosy: +cstratak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40698] distutils.command.upload md5_digest

[Donald Stufft]

Donald Stufft  added the comment:

> Does PyPI support other digests, e.g. SHA2-256 digest?

There is a simple and a complicated answer to this.

The simple answer is yes, PyPI supports uploads with any combination of MD5, 
SHA256, and blake2_256 (blake2b with a 256 digest, no personalization or key). 
It will also compute all 3 on an upload on it's own and verify that they match 
any provided hashes and to fill in any missing hashes.

The more complicated answer is the upload API is an old API from long before we 
started documenting and standardizing them, so when you start talking about non 
PyPI implementations of that API, what they support is kind of a big who knows.

More to the problem at hand:

We don't rely on this hash for security (We couldn't, it comes in the exact 
same payload as the artifact itself from the exact same source, someone who can 
modify the artifact en route can modify the hash too). So the inclusion of MD5 
is not a concern.

Removing it *might* break non-PyPI servers that attempted to implement this API 
and assumed it was a mandatory field (though I do not have any a priori 
knowledge of this being the case).

Adding additional hashes *might* break non-PyPI servers that assumed what 
distutils used to send was all it would ever send (this is unlikely though, 
most web tools ignore unknown form fields).

I looked into what twine is doing here, and it appears it is sending md5, 
sha256, and blake2_256 hashes all along with every request. However if FIPS 
mode has disabled MD5 it just skips generating and sending MD5 (but still sends 
the other two) and it appears it's done this for 2+ years.

It's probably safe to just mimc what twine is doing here, sending all 3 hashes, 
skip MD5 if it's unavailable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40698] distutils.command.upload md5_digest

[Christian Heimes]

Change by Christian Heimes :


--
nosy: +dstufft, eric.araujo, gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40698] distutils.command.upload md5_digest

[Christian Heimes]

New submission from Christian Heimes :

The distutils upload command creates a MD5 digest of the file content. This is 
not compatible with systems with systems that run under a strict security 
policy that blocks MD5.

Possible fixes are:

* declare that the MD5 digest is not used for security. Security is provided by 
TLS/SSL and HTTPS. The digest is just a simple checksum to detect file 
corruption during upload.
* Remove MD5 digest completely
* Don't create a MD5 digest if ``hashlib.md5(content)`` fails
* Skip the test case if MD5 is not available

Does PyPI support other digests, e.g. SHA2-256 digest?

--
components: Library (Lib)
messages: 369442
nosy: christian.heimes
priority: normal
severity: normal
status: open
title: distutils.command.upload md5_digest
type: behavior
versions: Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com