Re: [Numpy-discussion] NumPy 1.0.4 release
On 10/20/07, David Cournapeau [EMAIL PROTECTED] wrote: And I hope that in the end, scons will be used for numpy (for 1.1 ?), once I finish the conversion. I don't see situations where adding 350 Kb in the tarball can be an issue, so could you tell me what the problem would be ? Then if numpy is going to use scons for build itself in the near future, then there is no problem, and including scons in numpy tarball is good from the user side. Just a final observation. Try to make it possible to reuse a scons other than that included in numpy tarball. Packagers would be happy with this. Regards, -- Lisandro Dalcín --- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy 1.0.4 release
Travis E. Oliphant wrote: David Cournapeau wrote: Hi Jarrod, Would it be possible to merge some of the work I have done recently concerning cleaning configuration and so on (If nobody is against it, of course) ? If this is considerer too big of a change, what is the plan for a 1.1 release, if any ? Could you review what the disadvantages are for including your changes into the trunk? Would there be any code breakage?What is the approximate size increase of the resulting NumPy? I've tried to follow the discussion, but haven't kept up with all of it. There are been several things. I tried to keep different things separated for separate merge, and to make the review easier. cleanconfig branch == For the cleanconfig branch, I tried to be detailed enough in the ticket where I posted the patch: the goal is to separate the config.h into two separate headers, one public, one private. The point is to avoid polluting the C namespace when using numpy (the current config.h define HAVE_* and SIZEOF_* symbols which are used by autotools). This also makes it more similar to autoheader, so that if/when my scons work is integrated, replacing the custom config generators by scons will be trivial. The patch being actually generated from the branch, it does not really make sense to apply the patch, just merge the branch (I have also updated the code since I posted the patch). http://scipy.org/scipy/numpy/ticket/593 This is purely internal, this should not change anything to anyone on any platform, otherwise, it is a bug. numpy.scons branch == This is a much more massive change. Scons itself adds something like 350 kb to a bzip tarball. There are two milestones already in this branch. - The first one just adds some facilities to numpy.distutils, and it should not break nor change anything: this would enable people to build ctypes-based extension in a portable way, though. - The second milestone: I don't think it would be appropriate to merge this at this point for a 1.0.x release. Normally, nothing is used by default (e.g. you have to use a different setup.py to build with the new method), but it is so easy to screw things up that with such a short window (a few days), I would prefer avoiding potential breakage. The first milestone, if merged, consist in all revision up to 4177 (this one has been tested on many platforms, including Mac OS X intel, Windows with gnu compilers, linux with gnu compilers, and more obscure configurations as well; a breakage should be trivial to fix anyway). cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy 1.0.4 release
David Cournapeau wrote: Hi Jarrod, Would it be possible to merge some of the work I have done recently concerning cleaning configuration and so on (If nobody is against it, of course) ? If this is considerer too big of a change, what is the plan for a 1.1 release, if any ? Could you review what the disadvantages are for including your changes into the trunk? Would there be any code breakage?What is the approximate size increase of the resulting NumPy? I've tried to follow the discussion, but haven't kept up with all of it. Thanks, -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy 1.0.4 release
Travis E. Oliphant wrote: I've finally caught up with the discussion on aligned allocators for NumPy. In general I'm favorable to the idea, although it is not as easy to implement in 1.0.X because of the need to possibly change the C-API. The Python solution is workable and would just require a function call on the Python side (which one could call from the C-side as well with little difficulty, I believe Chuck Harris already suggested such a function). So, I think at least the Python functions are an easy addition for 1.0.4 (along with simple tests for alignment --- although a.ctypes.data % 16 is pretty simple and probably doesn't warrant a new function) I'm a bit more resistant to the more involved C-code in the patch provided with #568, because of the requested new additions to the C-API, but I understand the need. I'm currently also thinking heavily about using SIMD intrinsics in ufunc inner loops but will not likely get those in before 1.0.4. Unfortunately, all ufuncs that take advantage of SIMD instructions will have to handle the unaligned portions which may occur even if the start of the array is aligned, so the problem of thinking about alignment does not go away there with a simplified function call. I don't know anything about the ufunc machinery yet, but I guess you need to know the alignement of a given buffer, right ? This can be done easily. Actually, one problem I encountered (If I remember correctly) was that there is no pure C library facility in numpy: by that, I mean a simple C library, independant of python, which could be reusable by C code using numpy. For example, if we want to start thinking about using SIMD, I think it would be good to support basics in a pure C library. I don't see any downside to this approach ? A simple addition is an NPY_ALIGNED_16 and NPY_ALIGNED_32 flag for the PyArray_From_Any that could adjust the data-pointer as needed to get at least those kinds of alignment. We can't change the C-API for PyArray_FromAny to accept an alignment flag, and I'm pretty loath to do that even for 1.1. Is there a consensus? What do others think of the patch in ticket #568? Is there a need to add general-purpose aligned memory allocators to NumPy without a corresponding array_allocator? Having the NPY_ALIGNED_* flags would already be enough for most cases (for SIMD, you rarely, if ever needs more than 32 bytes alignment AFAIK). Those flags + general purposes memory allocators (in a C support library) would be enough to do everything we need to fft, for example. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy 1.0.4 release
Travis E. Oliphant wrote: We can't change the C-API for PyArray_FromAny to accept an alignment flag, and I'm pretty loath to do that even for 1.1. Ooops! Pleas read that as can't accept an alignment *argument* -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy 1.0.4 release
I've finally caught up with the discussion on aligned allocators for NumPy. In general I'm favorable to the idea, although it is not as easy to implement in 1.0.X because of the need to possibly change the C-API. The Python solution is workable and would just require a function call on the Python side (which one could call from the C-side as well with little difficulty, I believe Chuck Harris already suggested such a function). So, I think at least the Python functions are an easy addition for 1.0.4 (along with simple tests for alignment --- although a.ctypes.data % 16 is pretty simple and probably doesn't warrant a new function) I'm a bit more resistant to the more involved C-code in the patch provided with #568, because of the requested new additions to the C-API, but I understand the need. I'm currently also thinking heavily about using SIMD intrinsics in ufunc inner loops but will not likely get those in before 1.0.4. Unfortunately, all ufuncs that take advantage of SIMD instructions will have to handle the unaligned portions which may occur even if the start of the array is aligned, so the problem of thinking about alignment does not go away there with a simplified function call. A simple addition is an NPY_ALIGNED_16 and NPY_ALIGNED_32 flag for the PyArray_From_Any that could adjust the data-pointer as needed to get at least those kinds of alignment. We can't change the C-API for PyArray_FromAny to accept an alignment flag, and I'm pretty loath to do that even for 1.1. Is there a consensus? What do others think of the patch in ticket #568? Is there a need to add general-purpose aligned memory allocators to NumPy without a corresponding array_allocator? I would think the PyArray_FromAny and PyArray_NewFromDescr with aligned memory is more important which I think we could do with flag bits. -Travis O. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
