[issue18071] _osx_support compiler_fixup

[Roundup Robot]

Roundup Robot added the comment:

New changeset addd9210816b by Ned Deily in branch '2.7':
Issue #18071: Extension module builds on OS X could fail with TypeError
http://hg.python.org/cpython/rev/addd9210816b

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18071] _osx_support compiler_fixup

[Ralf Schmitt]

Changes by Ralf Schmitt :


--
nosy: +schmir

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18071] _osx_support compiler_fixup

[Alexey Borzenkov]

Alexey Borzenkov added the comment:

Also, to reproduce it you have to temporarily move away your /usr/bin/cc and 
/usr/bin/clang, which then triggers the affected code path.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18071] _osx_support compiler_fixup

[Alexey Borzenkov]

Alexey Borzenkov added the comment:

I was unaware of this issue (which homebrew works around incorrectly), and 
because of this I created issue 18580. The problem is indeed with unicode 
strings which are coming from _read_output in _osx_support. This is probably 
because this code was merged from Python 3, which in Python 2 it shouldn't use 
decode('utf-8') on command output.

--
nosy: +Alexey.Borzenkov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18071] _osx_support compiler_fixup

[Ned Deily]

Ned Deily added the comment:

Sorry, I am unable to reproduce the problem with a vanilla Python 2.7.5 and a 
downloaded Cython 0.19.1 either installing directly with Cython's setup.py or 
using Pip 1.3.1.  customize_compiler is used as part of every extension module 
build in Distutils; it is not something special. AFAICT, the compiler_so 
attribute of a CCompiler instance should always be a list.  It is intended to 
be set up as such by the set_executables method of CCompiler 
(Lib/distutils/ccompiler.py) which calls set_executable for each item, 
including compiler_so.

Now there could be a problem in Python 2 if, for some reason, the value for 
'compiler_so' has become a Unicode string: set_executable is looking only for 
an instance of type str.  It looks like the original value for 'compiler_so' is 
constructed in customize_compiler (in lib/distutils/sysconfig.py) by first 
concatenating CC + ' ' + CFLAGS, where the CC and CFLAGS values are obtained 
from the Python build (using get_config_vars) and possibly overridden by 
environment variables of the same names.  Then the value of the config var 
'CCSHARED' is appended; it most likely is empty.  So I would check to see the 
types of distutils.sysconfig.get_config_var("CC") and ("CFLAGS") and 
("CCSHARED") as well as os.environ["CC"] and ["CFLAGS"].  If any of them return 
a type unicode string, I think that could explain the behavior you see. The use 
of Unicode strings for these kinds of values are likely to lead to trouble, one 
way or another, in Python 2 for sure.  (Also, be aware that Cython extends Di
 stutils with its own build_ext to allow building of Cython modules. I took a 
very quick look at it and did not see any obvious problems.)

--
assignee: ronaldoussoren -> ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18071] _osx_support compiler_fixup

[Samuel John]

Samuel John added the comment:

The symptom for the end-user looks kind of weird:

running build_ext
building 'Cython.Plex.Scanners' extension
/ A p p l i c a t i o n s / X c o d e . a p p / C o n t e n t s / D e v e l o p 
e r / T o o l c h a i n s / X c o d e D e f a u l t . x c t o o l c h a i n / u 
s r / b i n / c l a n g   - f n o - s t r i c t - a l i a s i n g   - f n o - c 
o m m o n   - d y n a m i c   - I / h o m e b r e w / i n c l u d e   - I / h o 
m e b r e w / o p t / s q l i t e / i n c l u d e   - i s y s r o o t   / A p p 
l i c a t i o n s / X c o d e . a p p / C o n t e n t s / D e v e l o p e r / P 
l a t f o r m s / M a c O S X . p l a t f o r m / D e v e l o p e r / S D K s / 
M a c O S X 1 0 . 8 . s d k   - I / A p p l i c a t i o n s / X c o d e . a p p 
/ C o n t e n t s / D e v e l o p e r / P l a t f o r m s / M a c O S X . p l a 
t f o r m / D e v e l o p e r / S D K s / M a c O S X 1 0 . 8 . s d k / S y s t 
e m / L i b r a r y / F r a m e w o r k s / T k . f r a m e w o r k / V e r s i 
o n s / 8 . 5 / H e a d e r s   - D N D E B U G   - g   - f w r a p v   - O 3   
- W a l l   - W s t r i c t - p r o t 
 o t y p e s   
-I/homebrew/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/include/python2.7
 -c Cython/Plex/Scanners.c -o 
build/temp.macosx-10.8-x86_64-2.7/Cython/Plex/Scanners.o
unable to execute /: Permission denied
error: command '/' failed with exit status 1


Pasted here to help googlers to find it :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18071] _osx_support compiler_fixup

[Samuel John]

New submission from Samuel John:

In the `_osx_support.py` module, there seems to be a bug in the method 
`compiler_fixup` which occurs if 

* the `customize_compiler` method from `distutils/sysconfig` has been called 
and after that
* `_compile` from `distutils/unixcompiler.py` is called. The `_compile` method 
uses `_osx_support.compiler_fixup`.

The critical line in compiler_fixup is:

`compiler_so = list(compiler_so)`.

Usually `compiler_so` is a list like `['cc']` but not when `customize_compiler` 
(from sysconfig) has been called. The value returned by 
`sysconfig.get_config_var('LDSHARED')` is a unicode like for example in the 
case of a Python 2.7.5 built via Homebrew: 
  
"cc -bundle -undefined dynamic_lookup -L/homebrew/lib 
-L/homebrew/opt/sqlite/lib -isysroot 
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk".
When `list()` is applied to the value, it is split at each character and that 
cannot be intended.

I am not sure if I fully understand, but to fix this in `compiler_fixup`, 
instead of `compiler_so = list(compiler_so)`, I'd propose something along the 
lines of:

if isinstance(compiler_so, (str,unicode)):
compiler_so = compiler_so.split()

I found this by trying to `pip install cython` (it uses customize_compiler, I 
guess) on a homebrewed python.

To reproduce in an interactive shell we can emulate what is going on by calling 
`compiler_fixup` directly with the values sysconfig has stored. The exact 
string does not matter, because I think the any value returned by 
`sysconfig_get_config_var` is a (unicode) string and not a list:

import sysconfig
import _osx_support

_osx_support.compiler_fixup(sysconfig.get_config_var('LDSHARED'),sysconfig.get_config_var('CFLAGS'))


Not sure if other python versions are affected. I wasn't aware of 
`_osx_support` at all until now.

--
assignee: ronaldoussoren
components: Macintosh
messages: 190140
nosy: hynek, ned.deily, ronaldoussoren, samueljohn
priority: normal
severity: normal
status: open
title: _osx_support compiler_fixup
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com