[issue9189] Improve CFLAGS handling
Jed Brown added the comment:
Undefined variables are still a problem:
$ echo 'FROTZ = ${CFLAGS}' make.py3
$ python3
Python 3.3.1 (default, Apr 6 2013, 19:03:55)
[GCC 4.8.0] on linux
Type help, copyright, credits or license for more information.
import distutils.sysconfig
distutils.sysconfig.parse_makefile('make.py3')
Traceback (most recent call last):
File stdin, line 1, in module
File /usr/lib/python3.3/distutils/sysconfig.py, line 367, in parse_makefile
item = str(done['PY_' + n])
KeyError: 'PY_CFLAGS'
Proposed fix:
--- i/Lib/distutils/sysconfig.py
+++ w/Lib/distutils/sysconfig.py
@@ -357,7 +357,7 @@ def parse_makefile(fn, g=None):
found = False
else:
-item = str(done['PY_' + n])
+item = str(done.get('PY_' + n, ''))
else:
done[n] = item =
if found:
--
nosy: +jedbrown
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9189
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Éric Araujo mer...@netwok.org added the comment: I think this report can be closed now. -- status: pending - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Changes by Brett Cannon br...@python.org: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Stefan Krah stefan-use...@bytereef.org added the comment: Since I was the one who reopened this: The issues I found were fixed in r85358 and #9047 seems to be ok now. Setting to pending. -- resolution: - fixed stage: - committed/rejected status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Stefan Krah stefan-use...@bytereef.org added the comment: r85358 fixes the failures in test_sysconfig. The duplicate LDFLAGS are still present, but if it is generally accepted that make-LDFLAGS should be appended to configure-LDFLAGS, then I don't see any way around that. As for me, this could be closed again. Did Antoine's patch fix any of the other issues mentioned here? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Roumen Petrov bugtr...@roumenpetrov.info added the comment: -1 for the changes. Please restore to previous state. The patch break all uses of well documented build variables. -- nosy: +rpetrov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Éric Araujo mer...@netwok.org added the comment: Bugs that may be related: #4010 and #9437 Jeffrey, it would be great if you could have a look at those too. Thanks in advance. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Changes by Éric Araujo mer...@netwok.org: Removed file: http://bugs.python.org/file18062/smime.p7s ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Changes by Éric Araujo mer...@netwok.org: -- components: +Build nosy: +eric.araujo type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Stefan Krah stefan-use...@bytereef.org added the comment: This is what happens. In the Makefile, I have: CONFIGURE_LDFLAGS= -fprofile-arcs PY_LDFLAGS= $(CONFIGURE_LDFLAGS) $(LDFLAGS) But since LDFLAGS=-fprofile-arcs is in the environment, too, we get the duplication. The sysconfig issue is related. In Lib/sysconfig.py, _parse_makefile grabs LDFLAGS from the environment if it is set, so LDFLAGS ultimately expands to '-fprofile-arcs -fprofile-arcs'. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Ronald Oussoren ronaldousso...@mac.com added the comment: issue9047 is related to the issue Stefan mentions -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Stefan Krah stefan-use...@bytereef.org added the comment:
Starting with r82746 in py3k, I'm getting duplicate LDFLAGS in sysconfig:
make distclean
export BASECFLAGS=-ftest-coverage -fprofile-arcs
export LDFLAGS=-fprofile-arcs
./configure
make
$ ./python
Python 3.2a0 (py3k:82746M, Aug 16 2010, 00:25:49)
[GCC 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-23ubuntu3)] on linux2
Type help, copyright, credits or license for more information.
import sysconfig
sysconfig.get_config_var('LDFLAGS')
'-fprofile-arcs -fprofile-arcs'
test_sysconfig currently fails, so perhaps one of the duplicate flags
is mistakenly appended to LDFLAGS instead of LDSHARED:
$ ./python Lib/test/regrtest.py -uall test_sysconfig
[1/1] test_sysconfig
test test_sysconfig failed -- Traceback (most recent call last):
File /home/stefan/svn/py3k/Lib/test/test_sysconfig.py, line 285, in
test_ldshared_value
self.assertIn(ldflags, ldshared)
AssertionError: '-fprofile-arcs -fprofile-arcs' not found in 'gcc -pthread
-shared'
--
nosy: +skrah
resolution: fixed -
status: closed - open
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9189
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Ronald Oussoren ronaldousso...@mac.com added the comment:
On 9 Jul, 2010, at 20:55, Jeffrey Yasskin wrote:
Jeffrey Yasskin jyass...@gmail.com added the comment:
Oops. Thanks for telling me. Fixed in r82753.
I'm pretty sure this patch broke universal builds on OSX. Not the python build
itself, but building 3th-party extensions. I'll commit a fix later on.
In the Makefile:
LDSHARED= $(CC) $(LDFLAGS) -bundle -undefined dynamic_lookup
'gcc'
sysconfig.get_config_var('LDFLAGS')
'-arch i386 -arch ppc -arch x86_64 -isysroot /'
sysconfig.get_config_var('LDSHARED')
'gcc -bundle -undefined dynamic_lookup'
That is, the LDFLAGS aren't patched into the value of LDSHARED.
This is because LDFLAGS is actually PY_LDFLAGS in the makefile and the rename
from PY_LDFLAGS to LDFLAGS happens *after* variable expansion in sysconfig.
This doesn't affect the build of python itself because the Makefile explictly
sets LDFLAGS in the environment when running setup.py.
Ronald
--
Added file: http://bugs.python.org/file18062/smime.p7s
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9189
___
smime.p7s
Description: S/MIME cryptographic signature
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Jeffrey Yasskin jyass...@gmail.com added the comment: Oops, sorry about that. Is the fix to change LDSHARED to LDSHARED= $(CC) $(PY_LDFLAGS) -bundle -undefined dynamic_lookup ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Jeffrey Yasskin jyass...@gmail.com added the comment: Hearing no further comments, I've committed this as r82746. Let me know if it breaks anything. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Mark Dickinson dicki...@gmail.com added the comment: Jeffrey, the Linux buildbots are showing some distutils failures that look like they might be related to this change. For example: http://www.python.org/dev/buildbot/all/builders/amd64%20gentoo%203.x/builds/1351 test output contains: == ERROR: test_parse_makefile_base (distutils.tests.test_sysconfig.SysconfigTestCase) -- Traceback (most recent call last): File /home/buildbot/slave/py-build/3.x.norwitz-amd64/build/Lib/distutils/tests/support.py, line 16, in _capture_warnings return func(*args, **kw) File /home/buildbot/slave/py-build/3.x.norwitz-amd64/build/Lib/distutils/tests/test_sysconfig.py, line 58, in test_parse_makefile_base d = sysconfig.parse_makefile(self.makefile) File /home/buildbot/slave/py-build/3.x.norwitz-amd64/build/Lib/distutils/sysconfig.py, line 140, in parse_makefile return _sysconfig._parse_makefile(fn, g) File /home/buildbot/slave/py-build/3.x.norwitz-amd64/build/Lib/sysconfig.py, line 265, in _parse_makefile done[var] = done['PY_' + var] KeyError: 'PY_CFLAGS' -- nosy: +tarek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Jeffrey Yasskin jyass...@gmail.com added the comment: Oops. Thanks for telling me. Fixed in r82753. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
New submission from Jeffrey Yasskin jyass...@gmail.com: Patch at http://codereview.appspot.com/1749042. The idea here is to let the user set CFLAGS on either configure or make (or both), and have later settings appear later in the $CC command line. I left OPT, BASECFLAGS, and EXTRA_CFLAGS around in case people have written scripts using them, but I think they're superfluous as user-visible knobs after this patch. I prevented AC_PROG_CC from setting a default $CFLAGS value because the values it would set are already put into $BASECFLAGS when appropriate, and @CFLAGS@ needs to appear after @BASECFLAGS@ to allow the user to override Python's defaults at configure time. We could also accomplish this by removing BASECFLAGS and OPT entirely and instead prepending their contents to $CFLAGS in configure. That's a bigger patch, but if any of you feel strongly about it I can do that instead. I made the same changes for CPPFLAGS and LDFLAGS but no other user-settable variables. I don't have strong opinions about the exact set we support this for, as long as it includes CFLAGS, but these three seemed like a sensible set. -- messages: 109481 nosy: brett.cannon, jyasskin, lemburg, loewis, pitrou priority: normal severity: normal status: open title: Improve CFLAGS handling versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Changes by Mark Dickinson dicki...@gmail.com: -- nosy: +mark.dickinson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Brett Cannon br...@python.org added the comment: Just to say what I said in my Rietveld review of the patch: I support the approach Jeffrey is taking and the patch looks good. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Marc-Andre Lemburg m...@egenix.com added the comment: Jeffrey Yasskin wrote: New submission from Jeffrey Yasskin jyass...@gmail.com: Patch at http://codereview.appspot.com/1749042. The idea here is to let the user set CFLAGS on either configure or make (or both), and have later settings appear later in the $CC command line. I left OPT, BASECFLAGS, and EXTRA_CFLAGS around in case people have written scripts using them, but I think they're superfluous as user-visible knobs after this patch. Passing in user values to CFLAGS on configure should already work without the patch. I'm not sure why you'd want to modify CFLAGS on Makefile. I prevented AC_PROG_CC from setting a default $CFLAGS value because the values it would set are already put into $BASECFLAGS when appropriate, and @CFLAGS@ needs to appear after @BASECFLAGS@ to allow the user to override Python's defaults at configure time. We could also accomplish this by removing BASECFLAGS and OPT entirely and instead prepending their contents to $CFLAGS in configure. That's a bigger patch, but if any of you feel strongly about it I can do that instead. We've already had a long discussion about the use of AC_PROG_CC on another ticket. We are you trying to remove the default setting again ? BASECFLAGS is not supported by autoconf, it's a Python invention. Do you really want to put all the logic that autoconf uses to setup CFLAGS currently and in the future into our own configure code ? Are you willing to maintain that code ? We've also discussed switching the order of $(OPT) and $(CFLAGS): that doesn't work, since $(OPT) is meant to override the $(CFLAGS) settings. I made the same changes for CPPFLAGS and LDFLAGS but no other user-settable variables. I don't have strong opinions about the exact set we support this for, as long as it includes CFLAGS, but these three seemed like a sensible set. Some more comments: I don't really like the approach you've taken with the variable naming. It's better to stick to Makefile all caps standards and instead prepend Python specific variables with a PY or PY_ prefix. The already existing PY_CFLAGS should then be renamed to PY_RUNTIME_CFLAGS or just PYTHON_CFLAGS. Please also keep the default values for CFLAGS et al around, ie. CFLAGS = @CFLAGS@ PY_CFLAGS = $(BASECFLAGS) $(CFLAGS) $(OPT) $(EXTRA_CFLAGS) Note that using := will cause trouble with non-GNU makes. Why do you need this ? In summary: While we're breaking Makefile parsing anyway, we might as well go for the big patch and remove the whole BASECFLAGS, OPT and EXTRA_CFLAGS cruft. These cause all sorts of problem when building extensions for Python and not even distutils always gets them right (distutils needs to mimic the CFLAGS setup when setting up the compiler, so it has to replicate what's going on in the Makefile in distutils.utils.ccompiler.customize_compiler()). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Jeffrey Yasskin jyass...@gmail.com added the comment: Passing in user values to CFLAGS on configure should already work without the patch. Since the user's CFLAGS settings are overridden by Python's OPT settings, it doesn't already work without the patch. I'm not sure why you'd want to modify CFLAGS on Makefile. If I run `make clean; make CFLAGS=-Werror`, I expect that to add -Werror to the compilation line. I don't expect it to replace all of the flags configure set up, and I don't expect to have to re-run configure to add the -Werror. Doing this won't affect distutils' behavior and won't automatically cause all of Python to get recompiled, so it's not something people should do when installing Python, but it's helpful during development. We've already had a long discussion about the use of AC_PROG_CC on another ticket. We are you trying to remove the default setting again ? BASECFLAGS is not supported by autoconf, it's a Python invention. Do you really want to put all the logic that autoconf uses to setup CFLAGS currently and in the future into our own configure code ? Are you willing to maintain that code ? Given that AC_PROG_CC, in its entire history, has only grown the ability to check for -g and -O2, I'm not too worried. Our configure code already uses $ac_cv_prog_cc_g to take advantage of AC_PROG_CC's detection of -g, and we have more sophisticated logic around -O2 than it does. (Check the configure source if you don't believe that AC_PROG_CC's CFLAGS handling is pretty simple.) We've also discussed switching the order of $(OPT) and $(CFLAGS): that doesn't work, since $(OPT) is meant to override the $(CFLAGS) settings. It's meant to override AC_PROG_CC's bad CFLAGS settings. Once those are out of the way, we don't need to surprise users with a non-working CFLAGS anymore. I made the same changes for CPPFLAGS and LDFLAGS but no other user-settable variables. I don't have strong opinions about the exact set we support this for, as long as it includes CFLAGS, but these three seemed like a sensible set. Some more comments: I don't really like the approach you've taken with the variable naming. It's better to stick to Makefile all caps standards and instead prepend Python specific variables with a PY or PY_ prefix. The already existing PY_CFLAGS should then be renamed to PY_RUNTIME_CFLAGS or just PYTHON_CFLAGS. Sure. Please also keep the default values for CFLAGS et al around, ie. CFLAGS = @CFLAGS@ PY_CFLAGS = $(BASECFLAGS) $(CFLAGS) $(OPT) $(EXTRA_CFLAGS) This doesn't quite work, since the make command line overrides that CFLAGS assignment. I've saved the configure-provided values into CONFIGURE_CFLAGS, etc. Note that using := will cause trouble with non-GNU makes. Why do you need this ? http://www.openbsd.org/cgi-bin/man.cgi?query=make and http://www.freebsd.org/doc/en/books/pmake/variables.html say bsd make supports :=. I'm not sure where to find docs for other makes. I don't need := anymore, though, since I found another way to fix sysconfig. Removed. In summary: While we're breaking Makefile parsing anyway, we might as well go for the big patch and remove the whole BASECFLAGS, OPT and EXTRA_CFLAGS cruft. These cause all sorts of problem when building extensions for Python and not even distutils always gets them right (distutils needs to mimic the CFLAGS setup when setting up the compiler, so it has to replicate what's going on in the Makefile in distutils.utils.ccompiler.customize_compiler()). Makefile parsing isn't the only thing we care about compatibility with. I kept them around for people who've been setting them through build scripts or who are just used to setting them on the command line. Maybe they should go away too, but I think that's orthogonal to this patch, and should happen with a lag (maybe a whole release) to let people get used to CFLAGS first. That is, unless you can point out practical problems with the change around AC_PROG_CC. I've uploaded the new patch to http://codereview.appspot.com/1749042. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9189] Improve CFLAGS handling
Changes by Ronald Oussoren ronaldousso...@mac.com: -- nosy: +ronaldoussoren ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9189 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
