[issue36302] distutils creates unreproducible .so files

[STINNER Victor]

STINNER Victor  added the comment:

> Why do we need to sort those extensions by name?

Because calling sorted() is easy. Any other arbitrary choice would make sense. 
But giving the ability to build sources in a specific order, add dependencies, 
or something else, would a new features.

Also, distutils is now deprecated in the stdlib: the steering council approved 
https://www.python.org/dev/peps/pep-0632/

You should now report distutils issues to setuptools.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36302] distutils creates unreproducible .so files

[Matej Cepl]

Matej Cepl  added the comment:

> In particular, I sort the source list by age when working on larger 
> extensions, this makes sure that the source files I edited last get compiled 
> first, which speeds up edit/build cycles.

On the one hand, this sounds creepily like https://xkcd.com/1172, but on the 
other: why do we need to sort those extensions by name? Wouldn’t sorting by 
some attribute of those extensions (including their size) be better?

--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36302] distutils creates unreproducible .so files

[Ronald Oussoren]

Ronald Oussoren  added the comment:

That's the problem with any change to distutils, the API is unclear and that 
causes any change to be potentially breaking for existing users. That's why 
distutils has been mostly stagnant for years.

I have no particular wish w.r.t. changing behaviour, I've a local workaround 
for now and am hoping someone will write a tool similar to flit for packages 
that include extensions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36302] distutils creates unreproducible .so files

[STINNER Victor]

STINNER Victor  added the comment:

> An unfortunate side effect of this change is that changes the build order 
> even if the source list order is relevant. 

That sounds like a new feature. I don't think that it was supported previously. 
Or maybe it was more an implementation detail.

Please open a new issue if you would like to add the ability to specify in 
which order source files must be compiled.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36302] distutils creates unreproducible .so files

[Ronald Oussoren]

Ronald Oussoren  added the comment:

An unfortunate side effect of this change is that changes the build order even 
if the source list order is relevant. 

In particular, I sort the source list by age when working on larger extensions, 
this makes sure that the source files I edited last get compiled first, which 
speeds up edit/build cycles.

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36302] distutils creates unreproducible .so files

[STINNER Victor]

STINNER Victor  added the comment:

I'm not excited about backporting this change to Python 3.8 and older. Python 
3.8 also switch to its feature freeze for Python 3.8.0. This change is 
borderline between bugfix and feature. In case of doubt, I prefer to do nothing.

See concerns in the PR discussion:
https://github.com/python/cpython/pull/12341#issuecomment-473361985

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.9 -Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36302] distutils creates unreproducible .so files

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0d30ae1a03102de07758650af9243fd31211325a by Victor Stinner 
(Bernhard M. Wiedemann) in branch 'master':
bpo-36302: Sort list of sources (GH-12341)
https://github.com/python/cpython/commit/0d30ae1a03102de07758650af9243fd31211325a


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36302] distutils creates unreproducible .so files

[Bernhard M. Wiedemann]

Change by Bernhard M. Wiedemann :


--
keywords: +patch
pull_requests: +12308
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36302] distutils creates unreproducible .so files

[Bernhard M. Wiedemann]

New submission from Bernhard M. Wiedemann :

While working on reproducible builds for openSUSE, I found countless python 
modules that come with binary .so files that did not build reproducibly from 
non-deterministic filesystem readdir order.

One contributing factor is bpo-30461 that will not be fixed.
I found that a simple patch can be done in distutils
instead of fixing an infinite number of current and future python modules.

--
components: Distutils
messages: 337983
nosy: bmwiedemann, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: distutils creates unreproducible .so files
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com