On Tue, Jun 2, 2015 at 3:41 AM, Robert Collins <[email protected]> wrote: > On 2 June 2015 at 02:51, Erik Bray <[email protected]> wrote: >> On Sun, May 31, 2015 at 5:07 PM, AJ Friend <[email protected]> wrote: >>> Hi, >>> >>> I'm trying to write a new `setup.py` file for an extension module to >>> wrap a C library (https://github.com/cvxgrp/scs). >>> >>> The current `setup.py` file imports numpy. I'm trying to delay that >>> import statement until setuptools has a chance to install numpy if >>> it's not already installed. I'm trying to do that with this bit of >>> code: >>> >>> from setuptools.command.build_ext import build_ext as _build_ext >>> class build_ext(_build_ext): >>> def finalize_options(self): >>> _build_ext.finalize_options(self) >>> # Prevent numpy from thinking it is still in its setup process: >>> __builtins__.__NUMPY_SETUP__ = False >>> import numpy >>> self.include_dirs += ext['include_dirs'] + [numpy.get_include()] >>> >>> Running `python setup.py install` seems to work fine on my OSX >>> machine, but when I run `pip install .` in the directory with >>> `setup.py`, I get a clang error that it can't find one of the header >>> files. >>> >>> Any idea why that would be happening? Could it have anything to do >>> with the relative path I'm giving for the include directories? >>> >>> Also, I had trouble finding good documentation on subclassing >>> build_ext. Does anyone know if setting self.include_dirs overwrites or >>> appends to the include_dirs attribute of an Extension object defined >>> later in setup.py? >>> >>> For the curious, my current attempt at setup.py is >>> athttps://github.com/ajfriend/scs/blob/setup2/python/setup.py. The >>> original can be found in the same directory. >>> >>> More generally, since I'm new to python packaging, I'm not sure how >>> well or correctly I've written my `setup.py` file. Any feedback on >>> doing things correctly would be appreciated. >> >> Hi AJ, >> >> For a lot of things in Python packaging there is not, sadly, One Right >> Way to Do It. Your setup.py looks okay though. >> >> You may want to have a look at the get_numpy_include_path utility here: >> https://github.com/astropy/astropy-helpers/blob/7ee7e543641759ed1ee2b691bba1378cec76a001/astropy_helpers/utils.py#L66 >> >> It's similar to what you're already doing, but maybe a little more >> 'robust'. In particular, I think the reload of the numpy module may >> be important. > > It is, because what numpy is doing is importing from its own tree > before setup has completed - which is in general unsafe. > > And numpy may be in process depending on exactly what setuptools > setup_requires easy-install code has done, which can lead to the > symptoms you see. This should be better once we get pip interpreting > setup_requires for you, but a perhaps shorter term fix would be to > factor out the distutils support glue from numpy (which seems pretty > self contained) into a out of package file - either adjacent to > setup.py, or, if numpy is willing to make the dependency on setuptools > a hard dep (which I recommend), then as a separate package which can > be setup_require'd. > > That would avoid the bootstrapping complexity, resulting in numpy not > being imported until it has actually been installed, and things should > be rosy for folk downstream of numpy from that point on.
That might not be a bad idea. This is more or less what we did for Astropy with astropy-helpers (this was done also so that other software distributions in the Astropy affiliated package ecosystem could take advantage of it). I think maybe splitting out much of the stuff in numpy.distutils into its own package (though unfortunately it would need a new name since I don't think we could make numpy into a namespace package) might help. Though I'm not sure how that would work for numpy.get_includes(). Would it just figure out where the include files *would* be installed to if one were installing Numpy into the currently active environment? Erik _______________________________________________ Distutils-SIG maillist - [email protected] https://mail.python.org/mailman/listinfo/distutils-sig
