Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-09 Thread Charles R Harris
On 8/8/07, David Cournapeau [EMAIL PROTECTED] wrote: Charles R Harris wrote: Anne, On 8/8/07, *Anne Archibald* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: On 08/08/2007, Charles R Harris [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: On 8/8/07,

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-09 Thread Charles R Harris
On 8/9/07, Charles R Harris [EMAIL PROTECTED] wrote: On 8/8/07, David Cournapeau [EMAIL PROTECTED] wrote: Charles R Harris wrote: Anne, On 8/8/07, *Anne Archibald* [EMAIL PROTECTED] mailto: [EMAIL PROTECTED] wrote: On 08/08/2007, Charles R Harris [EMAIL PROTECTED]

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-09 Thread David Cournapeau
Charles R Harris wrote: Well, what you want might be very easy to do in python, we just need to check the default alignments for doubles and floats for some of the other compilers, architectures, and OS's out there. On the other hand, you might not be able to request a c malloc that is

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-09 Thread David Cournapeau
Charles R Harris wrote: Ah, you want it in C. What would be the use to get SIMD aligned arrays in python ? David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-09 Thread Charles R Harris
On 8/9/07, David Cournapeau [EMAIL PROTECTED] wrote: Charles R Harris wrote: Well, what you want might be very easy to do in python, we just need to check the default alignments for doubles and floats for some of the other compilers, architectures, and OS's out there. On the other hand,

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-09 Thread Charles R Harris
On 8/9/07, David Cournapeau [EMAIL PROTECTED] wrote: Charles R Harris wrote: Ah, you want it in C. What would be the use to get SIMD aligned arrays in python ? If I wanted a fairly specialized routine and didn't want to touch the guts of numpy, I would pass the aligned array to a C

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-09 Thread Stefan van der Walt
On Thu, Aug 09, 2007 at 04:52:38PM +0900, David Cournapeau wrote: Charles R Harris wrote: Well, what you want might be very easy to do in python, we just need to check the default alignments for doubles and floats for some of the other compilers, architectures, and OS's out there. On

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-09 Thread David Cournapeau
On 8/9/07, Stefan van der Walt [EMAIL PROTECTED] wrote: It doesn't really matter where the memory allocation occurs, does it? As far as I understand, the underlying fftw function has some flag to indicate when the data is aligned. If so, we could expose that flag in Python, and do something

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-08 Thread Stefan van der Walt
On Tue, Aug 07, 2007 at 01:33:24AM -0400, Anne Archibald wrote: Well, it can be done in Python: just allocate a too-big ndarray and take a slice that's the right shape and has the right alignment. But this sucks. Could you explain to me why is this such a bad idea? Stéfan

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-08 Thread Anne Archibald
On 08/08/2007, Stefan van der Walt [EMAIL PROTECTED] wrote: On Tue, Aug 07, 2007 at 01:33:24AM -0400, Anne Archibald wrote: Well, it can be done in Python: just allocate a too-big ndarray and take a slice that's the right shape and has the right alignment. But this sucks. Could you

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-08 Thread Charles R Harris
On 8/8/07, Anne Archibald [EMAIL PROTECTED] wrote: On 08/08/2007, Stefan van der Walt [EMAIL PROTECTED] wrote: On Tue, Aug 07, 2007 at 01:33:24AM -0400, Anne Archibald wrote: Well, it can be done in Python: just allocate a too-big ndarray and take a slice that's the right shape and has

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-08 Thread Anne Archibald
On 08/08/2007, Charles R Harris [EMAIL PROTECTED] wrote: On 8/8/07, Anne Archibald [EMAIL PROTECTED] wrote: Oh. Well, it's not *terrible*; it gets you an aligned array. But you have to allocate the original array as a 1D byte array (to allow for arbitrary realignments) and then align it,

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-08 Thread Charles R Harris
Anne, On 8/8/07, Anne Archibald [EMAIL PROTECTED] wrote: On 08/08/2007, Charles R Harris [EMAIL PROTECTED] wrote: On 8/8/07, Anne Archibald [EMAIL PROTECTED] wrote: Oh. Well, it's not *terrible*; it gets you an aligned array. But you have to allocate the original array as a 1D byte

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-08 Thread Matthieu Brucher
My 64 bit linux on Intel aligns arrays, whatever the data type, on 16 byte boundaries. It might be interesting to see what happens with the Intel and MSVC comipilers, but I expect similar results. According to the doc on the msdn, the data should be 16-bits aligned. Matthieu

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-08 Thread Charles R Harris
On 8/8/07, Matthieu Brucher [EMAIL PROTECTED] wrote: My 64 bit linux on Intel aligns arrays, whatever the data type, on 16 byte boundaries. It might be interesting to see what happens with the Intel and MSVC comipilers, but I expect similar results. According to the doc on the msdn, the

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-08 Thread David Cournapeau
Charles R Harris wrote: Anne, On 8/8/07, *Anne Archibald* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: On 08/08/2007, Charles R Harris [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: On 8/8/07, Anne Archibald [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-07 Thread Matthieu Brucher
For platforms without posix_memalign, I don't see how to implement a memory allocator with an arbitrary alignment (more precisely, I don't see how to free it if I cannot assume a fixed alignement: how do I know where the real pointer is ?). Visual Studio seems to offer a counter part (also

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-07 Thread David Cournapeau
Anne Archibald wrote: Well, it can be done in Python: just allocate a too-big ndarray and take a slice that's the right shape and has the right alignment. But this sucks. Stephen G. Johnson posted code earlier in this thread that provides a portable aligned-memory allocator - it handles the

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-07 Thread Charles R Harris
On 8/6/07, Anne Archibald [EMAIL PROTECTED] wrote: On 06/08/07, David Cournapeau [EMAIL PROTECTED] wrote: Well, when I proposed the SIMD extension, I was willing to implement the proposal, and this was for a simple goal: enabling better integration with many numeric libraries which need

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-06 Thread Lisandro Dalcin
On 8/3/07, David Cournapeau [EMAIL PROTECTED] wrote: Here is what I can think of: - adding an API to know whether a given PyArrayObject has its data buffer 16 bytes aligned, and requesting a 16 bytes aligned PyArrayObject. Something like NPY_ALIGNED, basically. - forcing data

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-06 Thread Anne Archibald
On 06/08/07, David Cournapeau [EMAIL PROTECTED] wrote: Well, when I proposed the SIMD extension, I was willing to implement the proposal, and this was for a simple goal: enabling better integration with many numeric libraries which need SIMD alignment. As nice as a custom allocator might be,

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-06 Thread David Cournapeau
Anne Archibald wrote: I have to agree. I can hardly volunteer David for anything, and I don't have time to implement this myself, but I think a custom allocator is a rather special-purpose tool; if one were to implement one, I think the way to go would be to implement a subclass of ndarray

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-06 Thread Anne Archibald
On 07/08/07, David Cournapeau [EMAIL PROTECTED] wrote: Anne, you said previously that it was easy to allocate buffers for a given alignment at runtime. Could you point me to a document which explains how ? For platforms without posix_memalign, I don't see how to implement a memory allocator

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-05 Thread Steven G. Johnson
On Aug 4, 3:24 am, Anne Archibald [EMAIL PROTECTED] wrote: It seems to me two things are needed: * A mechanism for requesting numpy arrays with buffers aligned to an arbitrary power-of-two size (basically just using posix_memalign or some horrible hack on platforms that don't have it).

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-04 Thread Charles R Harris
On 8/3/07, Charles R Harris [EMAIL PROTECTED] wrote: On 8/3/07, David Cournapeau [EMAIL PROTECTED] wrote: Andrew Straw wrote: Dear David, Both ideas, particularly the 2nd, would be excellent additions to numpy. I often use the Intel IPP (Integrated Performance Primitives)

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-04 Thread David Cournapeau
Here's a hack that google turned up: (1) Use static variables instead of dynamic (stack) variables (2) Use in-line assembly code that explicitly aligns data (3) In C code, use *malloc* to explicitly allocate variables Here is Intel's example of (2): ; procedure

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-04 Thread Anne Archibald
On 04/08/07, David Cournapeau [EMAIL PROTECTED] wrote: Here's a hack that google turned up: I'd avoid hacks in favour of posix_memalign (which allows arbitrary degrees of alignment. For one thing, freeing becomes a headache (you can't free a pointer you've jiggered!). - Check whether a

Re: [Numpy-discussion] numpy arrays, data allocation and SIMD alignement

2007-08-03 Thread Charles R Harris
On 8/3/07, David Cournapeau [EMAIL PROTECTED] wrote: Andrew Straw wrote: Dear David, Both ideas, particularly the 2nd, would be excellent additions to numpy. I often use the Intel IPP (Integrated Performance Primitives) Library together with numpy, but I have to do all my memory