On Wed, Aug 14, 2024 at 11:30:16AM +0200, Miro Hrončok wrote: > On 14. 08. 24 11:04, Zbigniew Jędrzejewski-Szmek wrote: > > On Wed, Aug 14, 2024 at 01:22:19AM +0200, Miro Hrončok wrote: > > > On 13. 08. 24 21:45, Zbigniew Jędrzejewski-Szmek wrote: > > > > On Tue, Aug 06, 2024 at 11:29:01AM +0200, Miro Hrončok wrote: > > > > > On 06. 08. 24 1:20, Miro Hrončok wrote: > > > > > > Hello Pythonistas. > > > > > > > > > > > > For years, the CFLAGS embedded in Python sysconfig contained -O2 in > > > > > > Fedora. This was how Python was built and by default, all flags > > > > > > used to > > > > > > build Python were embedded. > > > > > > > > > > > > Later, the flag was removed in Fedora 39 via this change: > > > > > > https://fedoraproject.org/wiki/Changes/Python_Extension_Flags_Reduction > > > > > > > > > > > > We wanted to remove as many flags as possible, with this motivation: > > > > > > > > > > > > """ > > > > > > Python developers will get more upstream-like experience when > > > > > > building > > > > > > Python extension modules and also closer to building vanilla C > > > > > > programs. > > > > > > """ > > > > > > > > > > > > Note that removing -O2 specifically was not the primary motivation, > > > > > > but > > > > > > the removal was intentional at that time. > > > > > > > > > > > > -------- > > > > > > > > > > > > Now we build Python 3.13 with -O3 via this change: > > > > > > https://fedoraproject.org/wiki/Changes/Python_built_with_gcc_O3 > > > > > > > > > > > > The change proposal said: > > > > > > > > > > > > """ > > > > > > Other Python extension modules will remain bulidng as before, e.g. > > > > > > in > > > > > > RPM packages, they will still be built with -O2... > > > > > > """ > > > > > > > > > > > > However, I made a mistake and the -O3 flag leaked into sysconfig > > > > > > CFLAGS. > > > > > > > > > > > > The good news is this does not seem to affect RPM packages, they are > > > > > > still being built with -O2, like this: > > > > > > > > > > > > <flags embedded in sysconfig> <$CFLAGS from automatic > > > > > > %set_build_flags> > > > > > > > > > > > > E.g. ... -fcf-protection -fexceptions -O3 -O2 -flto=auto ... > > > > > > > > > > > > The latter flag wins. > > > > > > > > > > > > ---- > > > > > > > > > > > > OTOH users building their own extension modules will get -O3. > > > > > > > > > > > > This is not what was intended. However, "Python developers will get > > > > > > more > > > > > > upstream-like experience" is more true now, because upstream Python > > > > > > builds use -O3: > > > > > > > > > > > > $ podman run --rm -ti python:3.12 /usr/bin/bash > > > > > > # python > > > > > > >>> import sysconfig > > > > > > >>> sysconfig.get_config_var('CFLAGS') > > > > > > '-fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall' > > > > > > > > > > > > > > > > > > So the question is: > > > > > > > > > > > > Do we keep -O3 for user-built extension modules for speed and > > > > > > upstream-like experience? (I would update the -O3 change proposal.) > > > > > > > > > > > > Or do we loose the flag, as currently documented? > > > > > > > > > > > > Alternatively, do something else entirely (e.g. embed -O2, or other > > > > > > flag...)? > > > > > > > > > > I slept on it and I support keeping the -O3 flag in sysconfig CFLAGS. > > > > > > > > > > The original motivation for Python_Extension_Flags_Reduction was: > > > > > > > > > > """ > > > > > Python developers will get more upstream-like experience when building > > > > > Python extension modules and also closer to building vanilla C > > > > > programs. > > > > > """ > > > > > > > > > > Keeping -O3 supports "more upstream-like experience". > > > > > Getting rid of it supports "closer to building vanilla C programs". > > > > > > > > > > When choosing from the two, I prefer the first one. > > > > > > > > > > Every time somebody pip installs something without a wheel, or even > > > > > builds > > > > > their own extension module code, on CI etc. they would benefit from > > > > > the > > > > > added speed. If we want Fedora to succeed on the CI field, we should > > > > > be > > > > > competitive. > > > > > > > > I think the argument about pip is important: with -O3, the users will > > > > get a "reasonable default". Local pip installs intended for computations > > > > are certainly better with '-O2' or '-O3' rather than the compiler > > > > default > > > > of -O0. > > > > > > > > But then the setup becomes inconsistent. The flags are: > > > > > > > > $ python -c "import sysconfig; > > > > print(sysconfig.get_config_var('CFLAGS'))" > > > > -fno-strict-overflow -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 > > > > -DNDEBUG > > > > -fcf-protection -fexceptions -fcf-protection -fexceptions > > > > -fcf-protection -fexceptions > > > > -O3 > > > > > > > > I'd argue that the goal is to have flags that provide a reasonable > > > > efficient default that integrates well with the cpython code provided > > > > by the distribution, > > > > -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer > > > > should be included too. (Those flags allow whole-system profiling of > > > > the system, and this works best if all of the system has frame pointers, > > > > so the extensions should have them too.) > > > > > > I'd argue that whoever wants to profile their system using frame pointers > > > is > > > also capable of setting the flags for the extension modules their build > > > themselves. > > > > Let me restate my argument. > > I think that there are two possibilities that make sense: > > in one, the flags that are exposed are the minimum that is required for > > ABI compatibility with the distro build. > > In the other, we additionally provide flags that give the most commonly > > useful and reasonable defaults. > > > > In the first case, the user has a very clear state, but they have to > > include additional flags to get "reasonable behaviour", e.g. efficiency. > > In the second case, things work nicely out-of-the-box, but custom and > > minimalistic builds require an additional step. I don't think it makes > > sense to mix the two, i.e. provide _some_ flags but not all flags for > > the common scenario. > > > > Benchmarking with frame pointers is a goal for both Fedora and the > > Python upstream. The whole system is built with FPs, and if the user > > tries to profile, that'll work as expected if all code is compiled > > appropriately. We can't expect the user to go back and recompile all > > code once they realize that some profiling is needed. (This is the > > same argument as in the discussions about distro builds.) > > If that's correct, the fix is to add the flags to %extension_cflags, right?
+1 > > Thus, if we go for the second case with -O3, the FP flags should be > > included too. > > > > (And if we go for the first case, at least > > '-Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG' should > > be dropped, because those things have no effect on the ABI.) > > I've checked and it's Python that inserts those, not us. Ack. I still think this should be changed, but if it's an upstream issue, we don't have to deal with it now. (In particular, defining -DNDEBUG is quite wrong. Disabling checks in downstream code is not something that should be done by Python.) Zbyszek -- _______________________________________________ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
