Re: [Numpy-discussion] New bug with setup,py develop
On Feb 11, 2008 9:24 PM, David Cournapeau [EMAIL PROTECTED] wrote: On Mon, 2008-02-11 at 20:42 -0700, Charles R Harris wrote: On Feb 11, 2008 7:10 PM, David Cournapeau [EMAIL PROTECTED] wrote: On Feb 11, 2008 5:40 PM, Charles R Harris [EMAIL PROTECTED] wrote: On Feb 11, 2008 1:21 AM, Robert Kern [EMAIL PROTECTED] wrote: I've just updated the SVN trunk to get the latest numscons merge. Something broke the support I put in for the setuptools develop command. In order to make sure that setuptools' develop works with numpy.distutils' build_src, we override the develop command to reinitialize the build_src command to add the --inplace option. This used to work as of r4772, but now any Fortran Extensions have the generated sources added twice. This causes links to fail since the same symbol shows up twice. While we're talking build, how do I set the compiler flags? Numpy here always compiles with -march=i386, which seems a bit conservative. My environment flags are also ignored, but I assume there is someway of getting the compile to behave. Well, you assumed wrong :) For some reason I was hoping you would pipe up :0) Yeah, that in itself is a good reason for trying something like scons. To be more exact: you can simply add flags in distutils. It is just that there is not much logic to handle different cases (e.g. having different set of warnings for pyrex vs swig vs normal C extensions is difficult). I note that with -O2 -finline-functions, or -O3, I can knock almost 30% off the string sort times. That's a lot better than I can do fooling around with the code. You should be able to do it just with numpy.distutils: CFLAGS=-O2 -finline-functions setup.py build Curiously, CFLAGS=-O3 -finline-functions causes the -fno-strict-aliasing flag to disappear when the random module is compiled, resulting in a lot of warnings and, in my experience, probably buggy code generation. So a safer bet is CFLAGS=-O3 -finline-functions -fno-strict-aliasing . Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New bug with setup,py develop
On Feb 11, 2008 3:49 AM, David Cournapeau [EMAIL PROTECTED] wrote: On Feb 11, 2008 5:38 PM, Robert Kern [EMAIL PROTECTED] wrote: On Feb 11, 2008 2:21 AM, Robert Kern [EMAIL PROTECTED] wrote: I've just updated the SVN trunk to get the latest numscons merge. Something broke the support I put in for the setuptools develop command. In order to make sure that setuptools' develop works with numpy.distutils' build_src, we override the develop command to reinitialize the build_src command to add the --inplace option. This used to work as of r4772, but now any Fortran Extensions have the generated sources added twice. Spoke too soon. It fails with r4772, too. Does it mean that it was already broken before numscons merge or not ? Yes. I think I figured out the problem. python setup.py develop works fine. However, I ran into the problem when I added develop to the end of a command line that already had build. So essentially, build_src gets run twice. I'm not sure there is anything that can be done about that given distutils' handling of options. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New bug with setup,py develop
On Feb 11, 2008 7:10 PM, David Cournapeau [EMAIL PROTECTED] wrote: On Feb 11, 2008 5:40 PM, Charles R Harris [EMAIL PROTECTED] wrote: On Feb 11, 2008 1:21 AM, Robert Kern [EMAIL PROTECTED] wrote: I've just updated the SVN trunk to get the latest numscons merge. Something broke the support I put in for the setuptools develop command. In order to make sure that setuptools' develop works with numpy.distutils' build_src, we override the develop command to reinitialize the build_src command to add the --inplace option. This used to work as of r4772, but now any Fortran Extensions have the generated sources added twice. This causes links to fail since the same symbol shows up twice. While we're talking build, how do I set the compiler flags? Numpy here always compiles with -march=i386, which seems a bit conservative. My environment flags are also ignored, but I assume there is someway of getting the compile to behave. Well, you assumed wrong :) For some reason I was hoping you would pipe up :0) Yeah, that in itself is a good reason for trying something like scons. I note that with -O2 -finline-functions, or -O3, I can knock almost 30% off the string sort times. That's a lot better than I can do fooling around with the code. It is one of the reasons why I started working on scons support. It is extremely ackward to control compilation flags with the python distutils from stdlib, and by design (flags are added at several different locations, which depend on the platform, and understanding the exact logic is beyond mere mortal capability). Thanks Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New bug with setup,py develop
On Mon, 2008-02-11 at 20:42 -0700, Charles R Harris wrote: On Feb 11, 2008 7:10 PM, David Cournapeau [EMAIL PROTECTED] wrote: On Feb 11, 2008 5:40 PM, Charles R Harris [EMAIL PROTECTED] wrote: On Feb 11, 2008 1:21 AM, Robert Kern [EMAIL PROTECTED] wrote: I've just updated the SVN trunk to get the latest numscons merge. Something broke the support I put in for the setuptools develop command. In order to make sure that setuptools' develop works with numpy.distutils' build_src, we override the develop command to reinitialize the build_src command to add the --inplace option. This used to work as of r4772, but now any Fortran Extensions have the generated sources added twice. This causes links to fail since the same symbol shows up twice. While we're talking build, how do I set the compiler flags? Numpy here always compiles with -march=i386, which seems a bit conservative. My environment flags are also ignored, but I assume there is someway of getting the compile to behave. Well, you assumed wrong :) For some reason I was hoping you would pipe up :0) Yeah, that in itself is a good reason for trying something like scons. To be more exact: you can simply add flags in distutils. It is just that there is not much logic to handle different cases (e.g. having different set of warnings for pyrex vs swig vs normal C extensions is difficult). I note that with -O2 -finline-functions, or -O3, I can knock almost 30% off the string sort times. That's a lot better than I can do fooling around with the code. You should be able to do it just with numpy.distutils: CFLAGS=-O2 -finline-functions setup.py build What should be easier with numscons at some point is a fine grained control. For example, the basic configuration is set in files similar to site.cfg right now, and I intend to add the possibility to set your own file in a near future (you can look for the file numscons/core/compiler.cfg to see how I do things now, in numscons sources). If that's the kind of things you are interested in, do not hesitate to try thing with numscons and tell me what's missing/what could be improved. I am mostly focused on getting things working by default on all platforms right now, but I am interested by inputs from other people with different needs. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New bug with setup,py develop
On Feb 11, 2008 5:40 PM, Charles R Harris [EMAIL PROTECTED] wrote: On Feb 11, 2008 1:21 AM, Robert Kern [EMAIL PROTECTED] wrote: I've just updated the SVN trunk to get the latest numscons merge. Something broke the support I put in for the setuptools develop command. In order to make sure that setuptools' develop works with numpy.distutils' build_src, we override the develop command to reinitialize the build_src command to add the --inplace option. This used to work as of r4772, but now any Fortran Extensions have the generated sources added twice. This causes links to fail since the same symbol shows up twice. While we're talking build, how do I set the compiler flags? Numpy here always compiles with -march=i386, which seems a bit conservative. My environment flags are also ignored, but I assume there is someway of getting the compile to behave. Well, you assumed wrong :) It is one of the reasons why I started working on scons support. It is extremely ackward to control compilation flags with the python distutils from stdlib, and by design (flags are added at several different locations, which depend on the platform, and understanding the exact logic is beyond mere mortal capability). You can use CFLAGS with distutils, but I don't really understand what's happening in this case: it seems that they are appended to the current compilation flags, but the order also seems to depend on the architecture. Sometimes they are inserted toward the end of compilation flags. With numscons, CFLAGS override the optimization/warning flags (but not -fPIC, etc... which are necessary to build the code; internally, optimization/debug/warning/necessary flags are separated). One example: with numpy.distutils, compiling for debug is not trivial: CFLAGS=-DDEBUG python setup.py build will still compile with -O2, and with -DNDEBUG. You need to add -O0 to override the default O2 used by distutils (at least for gcc, when several options of the same kind are given, the last one wins), and you have problems for options which cannot be overriden by later options. With numscons: CFLAGS=-DDEBUG will compile with only this option (and -fPIC, etc... on linux). I intend to add more options on the UI, but anway, they will be all overridable (for example, only overriding warning flags, etc...). cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New bug with setup,py develop
On Feb 11, 2008 5:38 PM, Robert Kern [EMAIL PROTECTED] wrote: On Feb 11, 2008 2:21 AM, Robert Kern [EMAIL PROTECTED] wrote: I've just updated the SVN trunk to get the latest numscons merge. Something broke the support I put in for the setuptools develop command. In order to make sure that setuptools' develop works with numpy.distutils' build_src, we override the develop command to reinitialize the build_src command to add the --inplace option. This used to work as of r4772, but now any Fortran Extensions have the generated sources added twice. Spoke too soon. It fails with r4772, too. Does it mean that it was already broken before numscons merge or not ? Nobody should be changed when numpy is built using setup.py, so any problem here is something I should fix ASAP. Supporting setuptools with numscons (using setupscons.py) is not a priority for me, though. David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] New bug with setup,py develop
I've just updated the SVN trunk to get the latest numscons merge. Something broke the support I put in for the setuptools develop command. In order to make sure that setuptools' develop works with numpy.distutils' build_src, we override the develop command to reinitialize the build_src command to add the --inplace option. This used to work as of r4772, but now any Fortran Extensions have the generated sources added twice. This causes links to fail since the same symbol shows up twice. David, any ideas? -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New bug with setup,py develop
On Feb 11, 2008 2:21 AM, Robert Kern [EMAIL PROTECTED] wrote: I've just updated the SVN trunk to get the latest numscons merge. Something broke the support I put in for the setuptools develop command. In order to make sure that setuptools' develop works with numpy.distutils' build_src, we override the develop command to reinitialize the build_src command to add the --inplace option. This used to work as of r4772, but now any Fortran Extensions have the generated sources added twice. Spoke too soon. It fails with r4772, too. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New bug with setup,py develop
On Feb 11, 2008 1:21 AM, Robert Kern [EMAIL PROTECTED] wrote: I've just updated the SVN trunk to get the latest numscons merge. Something broke the support I put in for the setuptools develop command. In order to make sure that setuptools' develop works with numpy.distutils' build_src, we override the develop command to reinitialize the build_src command to add the --inplace option. This used to work as of r4772, but now any Fortran Extensions have the generated sources added twice. This causes links to fail since the same symbol shows up twice. While we're talking build, how do I set the compiler flags? Numpy here always compiles with -march=i386, which seems a bit conservative. My environment flags are also ignored, but I assume there is someway of getting the compile to behave. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New bug with setup,py develop
On Feb 11, 2008 6:49 PM, David Cournapeau [EMAIL PROTECTED] wrote: On Feb 11, 2008 5:38 PM, Robert Kern [EMAIL PROTECTED] wrote: On Feb 11, 2008 2:21 AM, Robert Kern [EMAIL PROTECTED] wrote: I've just updated the SVN trunk to get the latest numscons merge. Something broke the support I put in for the setuptools develop command. In order to make sure that setuptools' develop works with numpy.distutils' build_src, we override the develop command to reinitialize the build_src command to add the --inplace option. This used to work as of r4772, but now any Fortran Extensions have the generated sources added twice. Spoke too soon. It fails with r4772, too. Does it mean that it was already broken before numscons merge or not ? Nobody should be changed when numpy is built using setup.py, so any ^^ I meant something, not nobody, here, obviously. David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
