[issue4508] distutils compiler not handling spaces in path to output/src files

[Steve Dower]

Steve Dower  added the comment:

Distutils is now deprecated (see PEP 632) and all tagged issues are being 
closed. From now until removal, only release blocking issues will be considered 
for distutils.

If this issue does not relate to distutils, please remove the component and 
reopen it. If you believe it still requires a fix, most likely the issue should 
be re-reported at https://github.com/pypa/setuptools

--
nosy: +steve.dower
resolution:  -> out of date
stage: test needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Greg Toombs]

Greg Toombs  added the comment:

Confirmed still broken in 3.6.3.

I ran into this when running CFFI. It does very wrong things with the path to 
cl.exe and include files with path spaces.

My workaround is to monkeypatch distutils.ccompiler.gen_preprocess_options such 
that include paths are formed as -I"path" and not -Ipath; and to monkeypatch 
distutils.spawn._nt_quote_args so that it leaves all of the arguments unquoted 
except the first arg (path to cl.exe), which I only got working by using 
tilde-mangled short names.

--
nosy: +Greg Toombs
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Mark Lawrence]

Changes by Mark Lawrence breamore...@yahoo.co.uk:


--
components:  -Distutils2
nosy: +dstufft
versions: +Python 3.4, Python 3.5 -3rd party, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Éric Araujo]

Éric Araujo added the comment:

Thanks Brian, let’s try and get this fixed.

 I've put together a patch adding the test requested. There is no problem on
 my Ubuntu machine with python 3.3.

Are you saying the test does not reproduce the bug discussed here?

 There is a comment in the file saying Don't load the xx module more than
 once, I am unsure whether my patch (using a renamed c file) violates this?

Hm I’m not quite sure if it’s enough that the extensions use different file 
names, or if they should also have different names inside the code.  Existing 
tests already create and import xx multiple times though…

 One can create a python file my file.py and can import it with
 __import__(my file). I couldn't do the same for a C extension.

One can’t do “import my file” though, so I would sweep this under the rug as an 
obscure corner case :‑)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Alexandru Gheorghe]

Alexandru Gheorghe added the comment:

I am having the same issue on Debian 7 x86-64 (3.2.0-4-amd64)
Python 2.7.3 (default, Jan  2 2013, 13:56:14) // [GCC 4.7.2] on linux2

Not noticing this bug I've opened a duplicate, for which the interesting 
comment can be found here: http://bugs.python.org/issue20824#msg212548

Please let me know if you need any files and I will try hard to provide some in 
my free time (also unittest from what I've read in the comments posted here).

--
nosy: +drunax

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Brian Thorne]

Brian Thorne added the comment:

I've put together a patch adding the test requested. There is no problem on my 
Ubuntu machine with python 3.3.
There is a comment in the file saying Don't load the xx module more than once, 
I am unsure whether my patch (using a renamed c file) violates this?


One extremely minor nitpick that I'll mention is inconsistency with spaces in 
the module name. One can create a python file my file.py and can import it 
with __import__(my file). I couldn't do the same for a C extension.

Building a C extension with distutils:

  module_name = 'ex ample'
  example_ext = Extension(module_name, [target_c])
  dist = Distribution({'name': module_name, 'ext_modules': [example_ext]})

Doesn't work due to the import mechanism looks for a PyInit function with the 
new module name (including a space):

  ImportError: dynamic module does not define init function (PyInit_ex ample)

I don't think this is worth making a ticket over.

--
Added file: http://bugs.python.org/file26700/distutils_space_test.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[fishdude]

fishdude fishd...@gmx.net added the comment:

i have to report that this issue also happens on Linux and Mac as well.
no matter wether python 2.6 or 2.7 is used.

--
components: +Macintosh
nosy: +fishdude
versions: +Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

Thanks for the confirmation.  Note that the versions field in this bug tracker 
indicates the versions that will get fixed, not all versions with the bug; 2.6 
only gets security fixes now.

Are you interested in updating the patch with a test (see my previous messages)?

--
assignee: tarek - eric.araujo
components: +Distutils2 -Extension Modules, Macintosh, Windows
nosy: +alexis
versions: +3rd party -Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

 Ok, I found a similar problem with MMTK.
I don’t know what that is.

 I am currently altering my distutils package to add a function called 
 nt_quote_dir
 that adds quotes to paths with spaces and then applies it to each path if the 
 platform
 is win32.
You shouldn’t have to do that; there’s already a similar function in 
distutils.spawn.

 When I'm done I will submit a diff after confirming that it works.
What is needed is a diff for distutils 2.7 or 3.2 adding a test with a file 
containing a space.  I can’t do anything with diffs for other projects.

--
versions: +Python 3.3 -Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[ipatrol]

ipatrol ipatrol6...@yahoo.com added the comment:

Ok, I found a similar problem with MMTK. I am currently altering my distutils 
package to add a function called nt_quote_dir that adds quotes to paths with 
spaces and then applies it to each path if the platform is win32. When I'm done 
I will submit a diff after confirming that it works.

--
components: +Extension Modules, Windows
nosy: +ipatrol

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Brian Thorne]

Brian Thorne hardb...@gmail.com added the comment:

Comments on the SciPy-Dev mailing list:


While testing Scipy's weave on several different Windows installations, I came
across some problems with spaces in paths that often prevent weave from working.
I can see a change that could probably get weave working on most Windows
installations, but it is a quick hack. Someone knowledgeable about distutils
(and numpy.distutils?) might be able to help me fix this properly. Below I
describe three common problems with weave on Windows, in the hope that this
information helps others, or allows someone to suggest how to fix the spaces-in-
paths problem properly.

I think there are three common problems that stop weave from working on Windows.
The first is not having a C compiler. Both Python(x,y) and EPD provide a C
compiler that seems to work fine, which is great!

The second problem is that if weave is installed to a location with a space in
the path, linking fails. There is already a scipy bug report about this
(http://projects.scipy.org/scipy/ticket/809). I've just commented on that
report, saying the problem appears to be with distutils, and there is already a
Python bug report about it (http://bugs.python.org/issue4508). Maybe someone
could close this scipy bug, or link it to the Python one somehow? In any case,
when using Python(x,y) or EPD, this bug will not show up if the default
installation locations are accepted. So, that's also good news!

The third problem is that if the Windows user name has a space in it (which in
my experience is quite common), compilation fails. Weave uses the user name to
create a path for its intermediate and compiled files. When the compilation
command is issued, the path with the space in it is also not quoted. Presumably
that is another error in distutils (or numpy.distutils)? Unfortunately I wasn't
able to pinpoint what function is failing to quote strings properly, because I
couldn't figure out the chain that leads to the compiler being called. However,
I can avoid the problem by removing spaces from the user name in weave itself
(catalog.py):

def whoami():
   return a string identifying the user.
   return (os.environ.get(USER) or os.environ.get(USERNAME) or
unknown).replace( ,)

(where I have added .replace( ,) to the existing code).



As well as that, I have another problem on another computer (which I
don't have access to right now) where I have a user name with a space in
it, and that causes several problems (although I think these problems
are the fault of distutils and nothing to do with scipy). I found a way
to work around that though by changing the environment variable for
USERNAME from Dan Goodman to DanGoo~1.



Unfortunately, it is still often necessary to patch weave to get it to work on 
Windows. If a Windows user name includes a space, Weave tries to use paths with 
spaces for its intermediate and compiled files. I have no idea if this is a 
problem with distutils (not quoting strings correctly), or if Weave itself 
should do something different, but I myself hack weave installations on Windows.


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

I tried using a space in a directory rather than in the file name and it 
worked. Thorney, can you repost your example as plain text files (separate 
source files or a diff against an empty dir) or at worst in an archive format 
readable by free software? Thank you.

(Don’t remember why I changed component, reverting.)

--
components: +Distutils -Distutils2
stage:  - unit test needed
versions:  -Python 2.5, Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Brian Thorne]

Brian Thorne hardb...@gmail.com added the comment:

Sorry, here is the example as a diff against an empty directory. 

(I'm 100% sure the Ubuntu archiving program that I just used is free software 
though...)

--
Added file: http://bugs.python.org/file18638/example.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

I can’t open the archive with unrar-free, and can’t apply the diff with patch 
(tried context mode and ed mode). Would you mind making it a unified diff this 
time?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Brian Thorne]

Changes by Brian Thorne hardb...@gmail.com:


Removed file: http://bugs.python.org/file18638/example.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Brian Thorne]

Brian Thorne hardb...@gmail.com added the comment:

Hopefully this works! Otherwise would a tar.gz or a zip archive be ok?

--
Added file: http://bugs.python.org/file18640/4508example.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

Thanks, this time it worked. I had to remove the example.i source file in 
setup.py since I don’t have swig installed, but apart from that, compiling “ex 
ample.c” or “ex ample/ex ample.c” worked.

Next steps:
- Write something similar to your example (can’t just copy it since Python is 
not under GPL) as a unit test for distutils; I’ll do it if nobody does it 
shortly.
- Confirm that the bug is present on windows and hunt for it (paths 
translation? argument quoting?)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
assignee: tarek - sjoerd
components: +Distutils2 -Distutils
nosy: +merwok, sjoerd
versions: +Python 2.5, Python 3.2 -Python 3.0

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

(Fixing damn HTML form changing select values, really sorry)

--
assignee: sjoerd - tarek
nosy:  -sjoerd

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[C. E. Ball]

C. E. Ball ceb...@users.sf.net added the comment:

I also found this bug while using SciPy's weave on Windows XP to compile
some inline c/c++ code.

Thorney's distutils_compiler_quoting.patch fixed the problem for me.


Here is the relevant part of the traceback (I hope):

Found executable C:\Program Files\Topographica\python_topo\mingw\bin\g++.exe
g++.exe:
c:\docume~1\ceball\locals~1\temp\ceball\python25_intermediate\compiler_
894ad5ed761bb51736c6d2b7872dc212\Release\program: No such file or directory
g++.exe:
files\topographica\python_topo\lib\site-packages\weave\scxx\weave_imp.o:
 No such file or directory


Chris

--
nosy: +ceball

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Tarek Ziadé]

Tarek Ziadé ziade.ta...@gmail.com added the comment:

I couldn't reproduce the problem yet, using your sample.

I tried to compile on Mac OSX (Python 2.5 GCC) and Windows XP (Python
2.5 , MinGW 5.1.4 and SWIG 1.2.3) and it worked well.

The output was right, (the .o file has a space, but the pyd/so files
come out properly), and I could use the gcd() function without any problem.

The call to the compiler quotes the c file correctly:

...
c:\Program Files\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall
-Ic:\Python25\include -Ic:\Python25\PC -c ex ample.c -o
build\temp.win32-2.5\Release\ex ample.o
...

This is done automatically with distutils.spawn._nt_quote_args under
Windows for instance. And this is basically what your patch is doing.

When does it crash exactly for you ? Do you have a traceback ?

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Brian Thorne]

Brian Thorne hardb...@gmail.com added the comment:

Ok - this bug has nothing to do with weave itself. I think any extension
written in C or C++ that has a space in the filename will raise the
error in both windows and gnu/linux.
Here is a simple c extension with and without spaces. The one with
spaces in the filename fails to build on standard python. Whereas the
one without should build fine. Did for me on windows 2000 on both python
2.5 and 2.6 with and without spaces in their paths.

Added file: http://bugs.python.org/file12693/python_extension_bug.rar

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Tarek Ziadé]

Tarek Ziadé ziade.ta...@gmail.com added the comment:

Hi. I am not familiar with weave. Could you provide a small sample of
code that raises this issue. This way, I will be able to write the
standalone test we can integrate in distutils together with your fix.

--
assignee:  - tarek
nosy: +tarek
priority:  - normal

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Martin v. Löwis]

Martin v. Löwis [EMAIL PROTECTED] added the comment:

The patch hasn't been accepted for the trunk, yet, so it is not a
candidate for a 2.5 backport.

--
nosy: +loewis
versions:  -Python 2.1.1, Python 2.1.2, Python 2.2, Python 2.2.1, Python 2.2.2, 
Python 2.2.3, Python 2.3, Python 2.4, Python 2.5, Python 2.5.3

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4508] distutils compiler not handling spaces in path to output/src files

[Brian Thorne]

New submission from Brian Thorne [EMAIL PROTECTED]:

I found this bug when using scipy's weave and having some inline C/C++
code. Weave uses distutils to compile the source in its own directory,
python was installed under program files. Unfortunately it doesn't
seem to handle spaces in the path to either the output or the temporary
source files.
I checked that this was the only problem by copying the weave directory
from site-packages and put it in another directory without spaces in the
path - weave and distutils then worked fine.
I then tried the g++ command seen in the stack trace at the command line
and got the same error - so altered the command by correctly quoting the
 -c path/to source/a.c and the -o objects.
This worked so set out changing it in distutils.

The two files I altered were unixcompiler.py (surprising as I am under
windows!) and cygwinccompiler.py 

This was clearly an older stable build so I checked out the python trunk
r67510 on windows (32bit), made a patch, and tested with my weave
dependant code.
(I am not familiar enough with distutils to make a standalone test).

The same method to cause a problem has the same effect in ubuntu - the
build directory seems to default to users home, but as the source path
has a space in it g++ gets confused.

--
components: Distutils
files: distutils_compiler_quoting.patch
keywords: patch
messages: 76848
nosy: Thorney
severity: normal
status: open
title: distutils compiler not handling spaces in path to output/src files
type: behavior
versions: Python 2.1.1, Python 2.1.2, Python 2.2, Python 2.2.1, Python 2.2.2, 
Python 2.2.3, Python 2.3, Python 2.4, Python 2.5, Python 2.5.3, Python 2.6, 
Python 2.7, Python 3.0, Python 3.1
Added file: http://bugs.python.org/file12213/distutils_compiler_quoting.patch

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue4508
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com