[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

Also this is due to an expected behaviour from gcc. From the documentation:

"If you use multiple -O options, with or without level numbers, the last such 
option is the one that is effective. "

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

Dug a bit further here.

The issue is that CFLAGS_NODIST will always come after normal CFLAGS (which are 
subsets of PY_CFLAGS and PY_CFLAGS_NODIST) [0][1].

The EXTRA_CFLAGS variable is appended at the end of PY_CFLAGS [2], hence as 
reported here, whatever compiler flag comes after, embedded within 
PY_CFLAGS_NODIST, will override the previous flags if they contradict each 
other.

So basically EXTRA_CFLAGS can be used only for flags that can't be overwritten 
by PY_CFLAGS_NODIST, which in my opinion, is not very useful in the context of 
just adding extra flags.

Commit adding the variable to Python 2.5: 
https://github.com/python/cpython/commit/08cd598c2145d00f1517c93cabf80a5d7d2a4bc0

"EXTRA_CFLAGS has been introduced as an environment variable to hold compiler 
flags that change binary compatibility"

Apparently it was added in order to avoid using the OPT variable for the 
various debug builds described in 
https://github.com/python/cpython/blob/master/Misc/SpecialBuilds.txt

On another note this flag will get passed down to distutils, so if it was used 
for building the interpreter, C extensions compiled by users will also inherit 
it.

Honestly I am not sure what the best solution would be here. If the various 
sub-debug special builds are still relevant and they stack, by doing for 
example $ make CFLAGS_NODIST="-DPy_TRACE_REFS" EXTRA_CFLAGS="-DPy_REF_DEBUG" 
then the issue can be closed, and the documentation can be more explicit that 
the EXTRA_CFLAGS is to be used only for the special builds.

If the EXTRA_CFLAGS can also be used for adding our own flags, then the flag 
handling needs to change to take this into account.

[0] https://github.com/python/cpython/blob/master/setup.py#L85
[1] https://github.com/python/cpython/blob/master/Makefile.pre.in#L115
[2] https://github.com/python/cpython/blob/master/Makefile.pre.in#L97

--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[STINNER Victor]

STINNER Victor  added the comment:

IMHO comments in Makefile.pre.in is a good start.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

I agree that documenting the flags is quite important, I've had a hard time 
trying to figure out how to implement the LDFLAGS_NODIST, and the change still 
broke macos builds (luckily it was fixed swiftly).

Nevertheless, this is still a bug which should be addressed.

On another note, where would be the best place to start documenting those 
flags? Makefile, the python docs, somewhere else?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[STINNER Victor]

STINNER Victor  added the comment:

Python has way too many variables to control compiler flags:
---
# Compiler options
OPT=@OPT@
BASECFLAGS= @BASECFLAGS@
BASECPPFLAGS=   @BASECPPFLAGS@
CONFIGURE_CFLAGS=   @CFLAGS@
# CFLAGS_NODIST is used for building the interpreter and stdlib C extensions.
# Use it when a compiler flag should _not_ be part of the distutils CFLAGS
# once Python is installed (Issue #21121).
CONFIGURE_CFLAGS_NODIST=@CFLAGS_NODIST@
# LDFLAGS_NODIST is used in the same manner as CFLAGS_NODIST.
# Use it when a linker flag should _not_ be part of the distutils LDFLAGS
# once Python is installed (bpo-35257)
CONFIGURE_LDFLAGS_NODIST=@LDFLAGS_NODIST@
CONFIGURE_CPPFLAGS= @CPPFLAGS@
CONFIGURE_LDFLAGS=  @LDFLAGS@
# Avoid assigning CFLAGS, LDFLAGS, etc. so users can use them on the
# command line to append to these values without stomping the pre-set
# values.
PY_CFLAGS=  $(BASECFLAGS) $(OPT) $(CONFIGURE_CFLAGS) $(CFLAGS) 
$(EXTRA_CFLAGS)
PY_CFLAGS_NODIST=$(CONFIGURE_CFLAGS_NODIST) $(CFLAGS_NODIST) 
-I$(srcdir)/Include/internal
# Both CPPFLAGS and LDFLAGS need to contain the shell's value for setup.py to
# be able to build extension modules using the directories specified in the
# environment variables
PY_CPPFLAGS=$(BASECPPFLAGS) -I. -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS) 
$(CPPFLAGS)
PY_LDFLAGS= $(CONFIGURE_LDFLAGS) $(LDFLAGS)
PY_LDFLAGS_NODIST=$(CONFIGURE_LDFLAGS_NODIST) $(LDFLAGS_NODIST)
NO_AS_NEEDED=   @NO_AS_NEEDED@
SGI_ABI=@SGI_ABI@
CCSHARED=   @CCSHARED@
# LINKFORSHARED are the flags passed to the $(CC) command that links
# the python executable -- this is only needed for a few systems
LINKFORSHARED=  @LINKFORSHARED@
ARFLAGS=@ARFLAGS@
# Extra C flags added for building the interpreter object files.
CFLAGSFORSHARED=@CFLAGSFORSHARED@
# C flags used for building the interpreter object files
PY_STDMODULE_CFLAGS= $(PY_CFLAGS) $(PY_CFLAGS_NODIST) $(PY_CPPFLAGS) 
$(CFLAGSFORSHARED)
PY_BUILTIN_MODULE_CFLAGS= $(PY_STDMODULE_CFLAGS) -DPy_BUILD_CORE_BUILTIN
PY_CORE_CFLAGS= $(PY_STDMODULE_CFLAGS) -DPy_BUILD_CORE
# Linker flags used for building the interpreter object files
PY_CORE_LDFLAGS=$(PY_LDFLAGS) $(PY_LDFLAGS_NODIST)
# Strict or non-strict aliasing flags used to compile dtoa.c, see above
CFLAGS_ALIASING=@CFLAGS_ALIASING@
---

I'm not sure which variables are "standard" and supposed to be overriden or 
not, because there is no documentation. The priority because these variables is 
not documented at all. For example:

* What the priority between BASECFLAGS, CFLAGS, OPT and EXTRA_FLAGS?
* Which flags are used to build Python core?
* Which flags are used to build modules of the standard library?
* Which flags are used to build third-party modules?

Linker flags are not documented neither which caused a regression for a short 
time (issue ith BLDSHARED if I recall correctly).

Examples of variables related to linker flags:

---
# LDFLAGS_NODIST is used in the same manner as CFLAGS_NODIST.
# Use it when a linker flag should _not_ be part of the distutils LDFLAGS
# once Python is installed (bpo-35257)
CONFIGURE_LDFLAGS_NODIST=@LDFLAGS_NODIST@
CONFIGURE_LDFLAGS=  @LDFLAGS@

PY_LDFLAGS= $(CONFIGURE_LDFLAGS) $(LDFLAGS)
PY_LDFLAGS_NODIST=$(CONFIGURE_LDFLAGS_NODIST) $(LDFLAGS_NODIST)
PY_CORE_LDFLAGS=$(PY_LDFLAGS) $(PY_LDFLAGS_NODIST)

LDSHARED=   @LDSHARED@ $(PY_LDFLAGS)
BLDSHARED=  @BLDSHARED@ $(PY_CORE_LDFLAGS)
---

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Miro Hrončok]

Change by Miro Hrončok :


--
nosy: +hroncok

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Miro Hrončok]

Change by Miro Hrončok :


--
keywords: +patch
pull_requests: +14781
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/15020

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Marcel Plch]

Marcel Plch  added the comment:

I believe you mean this downstream issue: 
https://bugzilla.redhat.com/show_bug.cgi?id=1712977

That issue is but only a consequence of a bad flag handling.
The bad flag handling affects not only test_gdb on that specific architecture, 
but the entire optimization level on all architectures, hence causing 
inconveniences in debugging in general on all architectures. It's a pure chance 
that one test caught this specific case.

It might also cause inconveniences with other use-cases for EXTRA_CFLAGS, as 
they might get overridden by CFLAGS_NODIST.

You can get the erroneous output by running the reproducer. That is:

$./configure
...
$ make CFLAGS_NODIST="-O2" EXTRA_CFLAGS="-Og"
gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall 
  -Og -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter 
-Wno-missing-field-initializers -Werror=implicit-function-declaration -O2 
-I./Include/internal  -I. -I./Include-DPy_BUILD_CORE -o Programs/python.o 
./Programs/python.c

-Og from EXTRA_CFLAGS gets overridden by -O2 from CFLAGS_NODIST.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

Could you provide more info, e.g. comparison between the proper and the 
erroneous output, as well as what it affects (test_gdb if I recall correctly)?

--
nosy: +cstratak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST

[Marcel Plch]

New submission from Marcel Plch :

Problem:
If you want to override CFLAGS by setting EXTRA_CFLAGS, they may have no effect 
if there are contrary flags in the CFLAGS_NODIST variable.

How to reproduce:
make CFLAGS_NODIST="-O2" EXTRA_CFLAGS="-Og"

If you look at GCC arguments, there is -O2 present *after* the -Og flag. This 
means -Og gets ignored.

--
components: Build
messages: 348168
nosy: Dormouse759
priority: normal
severity: normal
status: open
title: EXTRA_CFLAGS get overrided by CFLAGS_NODIST
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com