F41 Change Proposal - Python Built with gcc -03 (self-contained)
Wiki - https://fedoraproject.org/wiki/Changes/Python_built_with_gcc_O3
Discussion.fpo -
https://discussion.fedoraproject.org/t/f41-change-proposal-python-built-with-gcc-03-self-contained/112743
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Instead of
[https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags
Fedora's default `-O2` compiler flag], we will use `-O3` to build
CPython.
This only impacts the interpreter and Python standard library, not any
3rd party extension modules built as RPM or on developer machines.
This aligns with the way Python is built upstream.
According to our performance measurements, it makes Python
significantly faster (pyperformance geometric mean: 1.04x faster).
== Owner ==
* Name: [[User:churchyard|Miro Hrončok]]
* Email: mhron...@redhat.com
== Detailed Description ==
We will replace the `-O2` compiler flag with `-O3` when building the
python3.13 package. This change may be backported to older Pythons if
desired. [[Changes/Python3.13|Python 3.13 should be the main Python
version in Fedora 41+]].
The [https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags
Fedora packaging guidelines] about compiler flags explicitly say:
''> Overriding these flags for performance optimizations (for
instance, `-O3` instead of `-O2`) is generally discouraged. If you can
present benchmarks that show a significant speedup for this particular
code, this could be revisited on a case-by-case basis.''
This change proposal presents such benchmarks and a case for Python to
use `-O3`.
This change is limited to CPython interpreter and extension modules
from the Python standard library only thanks to
[[Changes/Python_Extension_Flags_Reduction]] (since Fedora 39). Other
Python extension modules will remain bulidng as before, e.g. in RPM
packages, they will still be built with `-O2`, unless Fedora changes
that globally. The extension modules built with `-O2` still work with
Python built with `-O3`.
== Feedback ==
== Benefit to Fedora ==
Upstream already builds Python with `-O3` by default. Fedora's Python
built with `-O3` is faster (1.04x):
{| class="wikitable sortable"
|+ Benchmark with python3.12-3.12.2-3.fc41
|-
! Benchmark!! -O2 !! -O3 !!
Change !! Significance
|-
| 2to3 || 465 ms || 446 ms ||
1.04x faster || Significant (t=21.72)
|-
| async_generators || 853 ms || 784 ms ||
1.09x faster || Significant (t=36.61)
|-
| async_tree_cpu_io_mixed || 1.19 sec || 1.11 sec ||
1.08x faster || Significant (t=13.38)
|-
| async_tree_cpu_io_mixed_tg || 1.17 sec || 1.09 sec ||
1.08x faster || Significant (t=18.69)
|-
| async_tree_eager || 202 ms || 189 ms ||
1.07x faster || Significant (t=7.99)
|-
| async_tree_eager_cpu_io_mixed|| 727 ms || 664 ms ||
1.09x faster || Significant (t=18.56)
|-
| async_tree_eager_cpu_io_mixed_tg || 633 ms || 558 ms ||
1.13x faster || Significant (t=24.53)
|-
| async_tree_eager_io || 1.72 sec || 1.68 sec ||
1.03x faster || Significant (t=6.13)
|-
| async_tree_eager_io_tg || 1.65 sec || 1.62 sec ||
1.02x faster || Significant (t=4.65)
|-
| async_tree_eager_memoization || 437 ms || 422 ms ||
1.04x faster || Significant (t=5.09)
|-
| async_tree_eager_memoization_tg || 330 ms || 322 ms ||
1.03x faster || Significant (t=2.60)
|-
| async_tree_eager_tg || 137 ms || 125 ms ||
1.09x faster || Significant (t=16.94)
|-
| async_tree_io|| 1.64 sec || 1.60 sec ||
1.02x faster || Significant (t=9.49)
|-
| async_tree_io_tg || 1.65 sec || 1.61 sec ||
1.02x faster || Not significant
|-
| async_tree_memoization || 895 ms || 871 ms ||
1.03x faster || Significant (t=3.73)
|-
| async_tree_memoization_tg|| 848 ms || 836 ms ||
1.01x faster || Not significant
|-
| async_tree_none || 718 ms || 700 ms ||
1.03x faster || Significant (t=6.90)
|-
| async_tree_none_tg || 686 ms || 659 ms ||
1.04x faster || Significant (t=13.11)
|-
| asyncio_tcp || 757 ms || 748 ms ||
1.01x faster || Not significant
|-
| asyncio_tcp_ssl || 2.58 sec || 2.56 sec ||
1.01x faster || Not significant
|-
| asyncio_websockets || 419 ms || 418 ms ||
1.00x faster || Not significant
|-
| bench_mp_pool|| 10.7 ms || 10.7 ms ||
1.00x faster || Not significant
|-
| bench_thread_pool|| 1.62 ms || 1.61 ms ||
1.01x faster || Not significant
|-
| chameleon
F41 Change Proposal - Reproducible Package Builds (System-Wide)
Wiki - https://fedoraproject.org/wiki/Changes/ReproduciblePackageBuilds Discussion.fpo - https://discussion.fedoraproject.org/t/f41-change-proposal-reproducible-package-builds-system-wide/112740 == Summary == A post-build cleanup is integrated into the RPM build process so that common causes of build irreproducibility in packages are removed, making most of Fedora packages reproducible. == Owner == * Name: Davide Cavalca * Name: Neil Hanlon * Name: [[User:churchyard|Miro Hrončok]] * Name: [[User:zbyszek|Zbigniew Jędrzejewski-Szmek]] * Email: dcava...@fedoraproject.org * Email: neil at shrug.pw * Email: mhroncok at redhat.com * Email: zbyszek at in.waw.pl == Detailed Description == As of 2023 there is an active effort to implement [https://docs.fedoraproject.org/en-US/reproducible-builds/ Reproducible builds] in Fedora. Reproducible builds will allow our users to be able to independently verify that the RPMs have not been tampered with (either maliciously or via hardware/software fault): someone can do an independent rebuild of a package and confirm that they get identical binaries when building with the same versions of the compiler and other tools. This Change allows us to move forward in this direction by removing the common sources of irreproducibility. [https://github.com/keszybz/add-determinism add-determinism] is a Rust program which, as its name suggests, adds determinism to files that are given as input by attempting to standardize metadata contained in binary or source files to ensure consistency and clamping to $SOURCE_DATE_EPOCH in all instances. `add-determinism` is the "Fedora version" of [https://salsa.debian.org/reproducible-builds/strip-nondeterminism strip-nondeterminism] from the Debian project. Since strip-nondeterminism is written in perl, it is undesirable for use in Fedora, as we don't want to pull perl in the buildroot for every package. It's worth noting that this Change does not intend to impose any specific reproducibility requirements on Fedora packages. Once this Change is implemented and we have been through a mass rebuild and can verify that the common causes of irreproducibility have indeed been removed, we can consider further steps. But that will be at least one release later. This change does add a small amount of time to the processing of RPMs at the end of a build. Accordingly, packages containing large quantities or sizes of files can be slower, but this effect is not expected to be noticeable. `add-determinism` takes steps to ensure it does not interfere with other buildroot post processors like `mangle-shebangs`, `python-hardlink`, `python-bytecompile`. It defaults to not doing any modifications in case it doesn't understand the input file or there are any other problems. A mechanism to opt-out will be provided: to either completely disable the postprocessing step or to disable specific "handlers" (i.e. implementations of cleanup for specific file types, for example static archives). See [https://github.com/keszybz/add-determinism/blob/main/rpm/macros.build-reproducibility macros.build-reproducibility]. === Related Changes === * [https://fedoraproject.org/wiki/Changes/ReproducibleBuildsClampMtimes Clamp build mtimes to SOURCE_DATE_EPOCH] * [https://fedoraproject.org/wiki/Changes/RPM-4.20 RPM 4.20] — this pulls in changes to `%autosetup -S git` which removed a source of irreproducibility. == Feedback == == Benefit to Fedora == Adding determinism (i.e., removing non-determinsim) enables the Fedora community to have confidence that, if given the same source code, build environment, build instructions, and metadata from the build artifacts, any party can recreate copies of the artifacts that are identical except for the signatures and some parts of metadata. Reproducibility of builds leads to packages of higher quality. It turns out that quite often those irreproducible bits are caused by an error or sloppiness in the code. In particular, any dependence on architecture in noarch packages is almost always unwanted and/or a bug. Test builds that check reproducibility will expose such instances. Reproducibility of builds makes it easier to develop packages: when a small change is made and a package is rebuilt (in the same environment), then with a reproducible package, the only difference is directly caused by the change. If the package is different every time it is rebuilt, making a comparison is much harder. Build reproducibility for noarch ''sub''packages solves the problem where package builds on different architectures are different, causing mock to reject the whole build. In particular, this issue occurs for [https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_byte_compilation_reproducibility pyc files]. This will now be solved without requiring opt-in from individual packages. == Scope == * Proposal Owners: ** Integrate `add-determinism` as a BuildRoot Policy script ** Add a dependency on `marshalparser` to `python3`
Orphaned packages looking for new maintainers
Report started at 2024-04-12 13:04:40 UTC The following packages are orphaned and will be retired when they are orphaned for six weeks, unless someone adopts them. If you know for sure that the package should be retired, please do so now with a proper reason: https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life Note: If you received this mail directly you (co)maintain one of the affected packages or a package that depends on one. Please adopt the affected package or retire your depending package to avoid broken dependencies, otherwise your package will be retired when the affected package gets retired. Request package ownership via the *Take* button in the left column on https://src.fedoraproject.org/rpms/ Full report available at: https://a.gtmx.me/orphans/orphans.txt grep it for your FAS username and follow the dependency chain. For human readable dependency chains, see https://packager-dashboard.fedoraproject.org/ For all orphaned packages, see https://packager-dashboard.fedoraproject.org/orphan Package (co)maintainersStatus Change container-workflow-tool orphan 2 weeks ago emacs-htmlize orphan 3 weeks ago jolokia-jvm-agent orphan 0 weeks ago kio-upnp-ms jgrulich, orphan 5 weeks ago libteam orphan 0 weeks ago loudgainorphan 4 weeks ago mingw-freeimage orphan 0 weeks ago mrxvt orphan 4 weeks ago nextcloud ichavero, orphan 2 weeks ago perl-WWW-Google-Contactsorphan 5 weeks ago php-aws-sdk3orphan 2 weeks ago php-bantu-ini-get-wrapper adamwill, orphan 2 weeks ago php-christophwurst-id3parserorphan 2 weeks ago php-deepdiver-zipstreamer orphan 2 weeks ago php-doctrine-dbal orphan, remi 2 weeks ago php-fgrosse-phpasn1 orphan 2 weeks ago php-giggsey-locale orphan 2 weeks ago php-guzzlehttp-guzzle6 orphan 2 weeks ago php-league-uri-interfaces orphan 2 weeks ago php-opencloud-openstack orphan 2 weeks ago php-opis-closureorphan, remi 2 weeks ago php-pimple orphan 2 weeks ago php-punic orphan 2 weeks ago php-ralouphie-getallheaders orphan 2 weeks ago php-scssphp orphan 2 weeks ago php-stecman-symfony-console-orphan 2 weeks ago completion prometheus-jmx-exporter orphan 0 weeks ago prometheus-simpleclient-javaorphan 0 weeks ago python-aiomqtt orphan 5 weeks ago python-autoprop orphan 5 weeks ago python-colorcet orphan 5 weeks ago python-jose orphan 1 weeks ago python-limits orphan 4 weeks ago python-paramorphan 5 weeks ago python-pyct orphan 5 weeks ago python-signature-dispatch orphan 5 weeks ago python-vecrec orphan 5 weeks ago snakeyaml mizdebsk, orphan, sbluhm 0 weeks ago vim-editorconfigorphan 1 weeks ago The following packages require above mentioned packages: Depending on: libteam (56), status change: 2024-04-07 (0 weeks ago) NetworkManager (maintained by: @gnome-sig, alexl, bengal, caolanm, danw, dcbw, ffmancera, ihuguet, liangwen12year, lkundrak, rhughes, rstrode, thaller) NetworkManager-1:1.46.0-2.fc41.src requires teamd-devel = 1.32-4.fc40 NetworkManager-team-1:1.46.0-2.fc41.x86_64 requires libteamdctl.so.0()(64bit) anaconda (maintained by: anaconda-maint, jkonecny, kkoukiou, m4rtink, rvykydal,
Fedora Linux 40 Final NO-GO
Due to outstanding blocker bugs[1], the Fedora Linux 40 Final RC -1.13 was declared NO-GO in today's meeting[2][3]. The next Fedora Linux 40 Final Go/No-Go meeting[4] will be held at 1700 UTC on Thursday 18th April in #meeting:fedoraproject.org on Matrix. The new target date for the F40 Final release is now Tuesday 23rd April. The schedule[5] has been updated accordingly. [1] https://qa.fedoraproject.org/blockerbugs/milestone/40/final/buglist [2] HTML Log: https://meetbot.fedoraproject.org/meeting_matrix_fedoraproject-org/2024-04-11/f40-final-go-no-go-meeting.2024-04-11-17.00.log.html [3] Text Log(minutes): https://meetbot.fedoraproject.org/meeting_matrix_fedoraproject-org/2024-04-11/f40-final-go-no-go-meeting.2024-04-11-17.00.txt [4] https://calendar.fedoraproject.org/meeting/10791/ [5] https://fedorapeople.org/groups/schedule/f-40/f-40-key-tasks.html -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney -- ___ devel-announce mailing list -- devel-announce@lists.fedoraproject.org To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
