[issue969718] BASECFLAGS are not passed to module build line

[STINNER Victor]

STINNER Victor  added the comment:

This issue has been fixed by bpo-36235.

--
nosy: +vstinner
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed
superseder:  -> distutils.sysconfig.customize_compiler() overrides CFLAGS var 
with OPT var if CFLAGS env var is set

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Stefan Krah]

Changes by Stefan Krah stefan-use...@bytereef.org:


--
nosy:  -skrah

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Ned Deily]

Ned Deily added the comment:

Since I just noticed this and haven't seen it mentioned already: for the 
record, the Python Makefile for current versions is affected by this issue.  
The sharedmods target, which calls setup.py to build the standard library 
shared modules, explicitly passes into Distutils via shell variables values for 
CC, LDSHARED, and OPT.  Unfortunately, as noted in this issue, Distutils does 
not look for an OPT variable.  So, while CC and LDSHARED can be overridden from 
the make command with either macro arguments or env vars, OPT cannot: only the 
value determined at configure time will be used.

I think that Chris's original distutils_opt_flag.patch should be applied to 
allow OPT to be overridden, without changing any other current behavior.  
AFAICT, the only compatibility issue would be if a script happened to already 
have an OPT env variable defined which would now get used by Distutils. I think 
the risks of that are pretty small and, in case, much smaller than the more 
extensive tweaking of Distutils behavior as is done in the Debian patches.

My interest in this comes from discovering that the OS X installer build script 
has been overriding OPT for its own purposes, thereby inadvertently dropping 
compiler options determined by configure (things like -fwrapv) which can result 
in a broken interpreter.  I've changed the installer build to no longer do 
that.  But that does mean that users of the OS X installer will now see those 
missing compiler options during extension module builds and it is conceivable 
that could cause problems for some ext modules and there wouldn't be a simple 
way to work around them (e.g. by setting an OPT env value).  If no one has any 
strong objections, I'll plan to commit that patch soon.

--
components:  -Distutils2
nosy: +ned.deily
versions:  -3rd party, Python 3.1, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[koobs]

Changes by koobs koobs.free...@gmail.com:


--
nosy: +koobs

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Arfrever Frehtes Taifersar Arahesis]

Arfrever Frehtes Taifersar Arahesis added the comment:

OPT should not be used in Distutils at all.

Lib/distutils/sysconfig.py should have:
if 'CFLAGS' in os.environ:
cflags = os.environ['CFLAGS']

Makefile.pre.in should have:
$(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' CFLAGS='$(PY_CFLAGS)' \
_TCLTK_INCLUDES='$(TCLTK_INCLUDES)' _TCLTK_LIBS='$(TCLTK_LIBS)' \
$(PYTHON_FOR_BUILD) $(srcdir)/setup.py $$quiet build

PY_CFLAGS is defined as:
PY_CFLAGS=  $(BASECFLAGS) $(OPT) $(CONFIGURE_CFLAGS) $(CFLAGS) 
$(EXTRA_CFLAGS)

So then OPT could be overriden when calling `make`.

See my patch for Distutils in bug #1222585. That patch also cleans handling of 
flags.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Ned Deily]

Ned Deily added the comment:

Arfrever, your proposal is certainly one of many possible solutions and one 
that would be appropriate if we were designing the Python configure, Makefile, 
and Distutils components from scratch or looking at major changes to Distutils. 
 But we're not at this point.  We all know that Distutils is brittle and I 
think at this point in its life it is best to change as little as possible 
without really good reason.  For better or worse, most distributors and package 
maintainers have figured out how to make Distutils do what they need to do 
until the next generation of package building starts being addressed post-3.4.  
The proposed patch solves a very specific problem with very little risk (IMO) 
to breaking anyone's current solutions.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Brett Cannon]

Changes by Brett Cannon br...@python.org:


--
nosy:  -brett.cannon
versions: +Python 3.3, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Stefan Krah]

Stefan Krah stefan-use...@bytereef.org added the comment:

Chris Lambacher rep...@bugs.python.org wrote:
 I was cross-compiling to a target architecture that does not support
 the -march option so I would not be able to provide a different one
 as override. 

Just out of curiosity: You are cross-compiling a C-extension? How can
you be sure that all values in pyconfig.h are correct?

 The debian-sysconfig-flags looks the most correct because it allows
 override at any point by emulating the CFLAGS = $(BASECFLAGS) $(OPT)
 $(EXTRA_CFLAGS) logic.
 
 It still looks like it needs an override for EXTRA_CFLAGS and the
 logic for LD_SHARED looks wrong. In the Makefile LDSHARED is just:
 LDSHARED=   $(CC) -shared -Wl,-O1 -Wl,-Bsymbolic-functions

I think Antoine is correct: EXTRA_CFLAGS show up as an empty string in
sysconfig (even if passed to configure and make), so there is no need
to override them.

As an aside, why do EXTRA_CFLAGS still exist in py3k when CFLAGS
can already be passed to configure and make?

I agree that the ldshared logic looks wrong. It is possible to specify
LDFLAGS via the environment, so why append CFLAGS. Changing this could
break existing build scripts though.

However, I don't see how the Debian patch could break anything, unless
someone relies on the fact that BASECFLAGS are suppressed in case CFLAGS
are given. I think the odds of this are virtually zero.

Any interest in a cleaned up version of the Debian patch (remove
double opt, line length, add documentation for overriding BASECFLAGS
and OPT)?

I'm asking, since I'm unsure about the degree of frozenness of distutils.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Chris Lambacher]

Chris Lambacher ch...@kateandchris.net added the comment:

Antoine said:
 I don't understand how you can cross-compile using the host Python
 Makefile. Could you explain?

The get_config_vars() function parses the host Makefile to get the values that 
we are discussing overriding.

 EXTRA_CFLAGS is not defined in the Makefile so it would probably be
 taken from the environment anyway (if my understand of Makefiles is
 right). That seems to explain the name EXTRA_CFLAGS, but I'm no
 specialist of the jungle that have building systems become under Unix.

You are correct, EXTRA_CFLAGS is not in the Makefile, but does provide us with 
an expected order of evaluation. i.e. Distutils is generating the equivalent 
command lines from the pieces and we should be respecting the Makefile order 
for consistency. Also, I don't know if there is a way to insert EXTRA_CFLAGS as 
an option to the ./configure script when Python is built such that it shows up 
in the resulting Makefile. If there is no way to get EXTRA_CFLAGS into the 
Makefile, then we should take it entirely out of the equation in the 
construction of the cflags var. 

Stephan Said:
 Just out of curiosity: You are cross-compiling a C-extension? How can
 you be sure that all values in pyconfig.h are correct?
That is the whole point of this discussion, if you can sub out the build 
parameters, you can insert the path to the .h files for the embedded target 
before any host specific .h files.

In order to build a C extension, you pretty much have to do it through 
Distutils and therefore the host python interpreter or else do a lot of extra 
work to get the right settings for each individual extension.
See the following for some basic how-to info: 
http://whatschrisdoing.com/blog/2006/10/06/howto-cross-compile-python-25/
http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

 Any interest in a cleaned up version of the Debian patch (remove
 double opt, line length, add documentation for overriding BASECFLAGS
 and OPT)?

I think it would be nice, but it should have a test if possible.
Otherwise there'll keep being regressions.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Stefan Krah]

Stefan Krah stefan-use...@bytereef.org added the comment:

EXTRA_CFLAGS were removed in r38848. Is it necessary to configure
OPT, BASECFLAGS and EXTRA_CFLAGS via the environment? I think it
might be sufficient to take CFLAGS from the Makefile and append
the environment CFLAGS.


Chris, would the new minimal patch solve your problem, too?

--
Added file: http://bugs.python.org/file20698/sysconfig-flags-minimal.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Chris Lambacher]

Chris Lambacher ch...@kateandchris.net added the comment:

I am not convinced that the minimal patch would work for my original issue. I 
wanted to be able to override the -march option which shows up in OPT on 
Fedora. I was cross-compiling to a target architecture that does not support 
the -march option so I would not be able to provide a different one as 
override. 

The proposed minimal patch would leave the OPT value and make it unchangeable 
because CFLAGS would pull out a value for OPT from the Makefile which shows as 
in my current Ubuntu system:

OPT=-DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
BASECFLAGS=  -fno-strict-aliasing
CFLAGS= $(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)

The debian-sysconfig-flags looks the most correct because it allows override at 
any point by emulating the CFLAGS = $(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS) logic.

It still looks like it needs an override for EXTRA_CFLAGS and the logic for 
LD_SHARED looks wrong. In the Makefile LDSHARED is just:
LDSHARED=   $(CC) -shared -Wl,-O1 -Wl,-Bsymbolic-functions

Why does LDSHARED need CFLAGS and CPPFLAGS? When provided as an override but 
not in the Makefile? Do we need to allow for this override in the case of 
overriding OPT, BASEFLAGS or EXTRA_CFLAGS?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

 I am not convinced that the minimal patch would work for my original
 issue. I wanted to be able to override the -march option which shows
 up in OPT on Fedora. I was cross-compiling to a target architecture
 that does not support the -march option so I would not be able to
 provide a different one as override. 

I don't understand how you can cross-compile using the host Python
Makefile. Could you explain?

 The proposed minimal patch would leave the OPT value and make it
 unchangeable because CFLAGS would pull out a value for OPT from the
 Makefile which shows as in my current Ubuntu system:
 
 OPT=-DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
 BASECFLAGS=  -fno-strict-aliasing
 CFLAGS= $(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)
 
 The debian-sysconfig-flags looks the most correct because it allows
 override at any point by emulating the CFLAGS = $(BASECFLAGS) $(OPT)
 $(EXTRA_CFLAGS) logic.
 
 It still looks like it needs an override for EXTRA_CFLAGS

EXTRA_CFLAGS is not defined in the Makefile so it would probably be
taken from the environment anyway (if my understand of Makefiles is
right). That seems to explain the name EXTRA_CFLAGS, but I'm no
specialist of the jungle that have building systems become under Unix.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Stefan Krah]

Stefan Krah stefan-use...@bytereef.org added the comment:

Éric, the Debian patch looks good to me and it solves my build problem.

The only question I have is why EXTRA_CFLAGS still go behind CFLAGS
and cannot be overridden via the environment.

But as it is, the patch is an improvement. I'm attaching the version
for 2.7.

--
stage:  - patch review
Added file: http://bugs.python.org/file20689/sysconfig-flags.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Why is OPT duplicated in get_config_vars(...)?
Why do OPT and BASECFLAGS environ vars override their Makefile values instead 
of accumulating with them?
Why is EXTRA_CFLAGS not configurable through environ?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Stefan Krah]

Stefan Krah stefan-use...@bytereef.org added the comment:

Antoine Pitrou rep...@bugs.python.org wrote:
 Why is OPT duplicated in get_config_vars(...)?

I missed that, thanks.

 Why do OPT and BASECFLAGS environ vars override their Makefile values
 instead of accumulating with them?

I think it would go too far to append in three places. If the environment
CFLAGS go to the end, everything can be overridden with a single variable.

 Why is EXTRA_CFLAGS not configurable through environ?

I don't know. Ideally the Debian people would comment if they had any
reasons for that. For me it would be sufficient if CFLAGS were configurable
without deleting OPT, BASECFLAGS or EXTRA_CFLAGS.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

Pythons in Debian seem to be immune to this problem, thanks to this patch: 
http://deb.li/3ku1g (via 
http://lists.debian.org/debian-python/2010/12/msg5.html).

I haven’t had time to learn the intricacies of make variables yet, so I can’t 
approve a patch.  I’m adding people from the nosy lists of other make variables 
bugs, hopefully someone will be able to review.

--
nosy: +brett.cannon, georg.brandl, jyasskin, lemburg, lukasz.langa, 
mark.dickinson, pitrou, ronaldoussoren, skrah

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

Duplicate report from Stephan Krah:

When CFLAGS are set, distutils drops -fno-strict-aliasing (among other
flags):


$ python2.7 setup.py build
...
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall 
-Wstrict-prototypes -fPIC -I./src -I/usr/local/include 
-I/usr/local/include/python2.7 -c src/gmpy.c -o 
build/temp.linux-x86_64-2.7/src/gmpy.o
...

$ CFLAGS=-fomit-frame-pointer python2.7 setup.py build
...
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes 
-fomit-frame-pointer -fPIC -I./src -I/usr/local/include 
-I/usr/local/include/python2.7 -c src/gmpy.c -o 
build/temp.linux-x86_64-2.7/src/gmpy.o
src/gmpy.c: In function ‘_cmp_to_object’:
src/gmpy.c:4692: warning: dereferencing type-punned pointer will break 
strict-aliasing rules
...

I'm not sure if this is intentional. The documentation says:

Compiler flags can also be supplied through setting the CFLAGS
environment variable. If set, the contents of CFLAGS will be added
to the compiler flags specified in the Setup file.

To me, this sounds as if they should be appended to the regular flags.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Jakub Wilk]

Changes by Jakub Wilk jw...@jwilk.net:


--
nosy: +jwilk

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Chris Lambacher]

Chris Lambacher ch...@kateandchris.net added the comment:

I am attaching a preliminary patch to allow override of $(OPT). I am not sure 
this is sufficient, but am wary of breaking packages that depend on the 
existing behaviour. 

The logic indeed seems wrong, but maybe this is something that has to go in 
distutils2 rather than distutils.

--
keywords: +patch
Added file: http://bugs.python.org/file19750/distutils_opt_flag.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
assignee:  - tarek
components: +Distutils2
nosy: +eric.araujo
versions: +3rd party, Python 3.2 -Python 2.6, Python 3.0

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Chris Lambacher]

Chris Lambacher ch...@kateandchris.net added the comment:

I am running into a problem related to this. I am attempting to cross
compile extensions but Fedora includes -march in the OPT variable. Since
there is no way to exclude the OPT values the build fails.

It seems that forcing OPT to stay the same is a disservice generally
speaking since as Marien said you potentially get extra conflicting
flags (-O seems like a stand-out since that could mess with gdb).

2.6 and 3.1 say:
if 'CFLAGS' in os.environ:
cflags = opt + ' ' + os.environ['CFLAGS']
ldshared = ldshared + ' ' + os.environ['CFLAGS']

There is no ability to override opt. I'd be willing to put together a
patch to allow opt to be overridden by an environment variable if I
could get some support for this from a core maintainer.

I am also open to other suggestions about how to get around this. I
potentially have time to put into fixing this (or the more general cross
compile issue).

--
nosy: +lambacck

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Roumen Petrov]

Changes by Roumen Petrov bugtr...@roumenpetrov.info:


--
nosy: +rpetrov

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Akira Kitada]

Changes by Akira Kitada akit...@gmail.com:


--
nosy: +tarek
type:  - behavior
versions: +Python 2.6, Python 2.7, Python 3.0, Python 3.1 -Python 2.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue969718
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com