[Numpy-discussion] [RFC] complex functions in npymath

2009-10-30 Thread Pauli Virtanen
Hi (esp. David), If there are no objections, I'd like to move Numpy's complex-valued C99-like functions to npymath: http://github.com/pv/numpy-work/tree/npymath-complex This'll come useful if we want to start eg. writing Ufuncs in Cython. I'm working around possible

Re: [Numpy-discussion] [RFC] complex functions in npymath

2009-10-30 Thread David Cournapeau
Hi Pauli, Pauli Virtanen wrote: Hi (esp. David), If there are no objections, I'd like to move Numpy's complex-valued C99-like functions to npymath: http://github.com/pv/numpy-work/tree/npymath-complex This'll come useful if we want to start eg. writing Ufuncs in Cython.

Re: [Numpy-discussion] [RFC] complex functions in npymath

2009-10-30 Thread Pauli Virtanen
Fri, 30 Oct 2009 18:34:07 +0900, David Cournapeau wrote: [clip] Actually, I am in the process of cleaning my numpy branches for review, and intend to push them into svn as fast as possible. Complex is pretty high on the list. Great! The missing piece in complex support in npymath is mostly

Re: [Numpy-discussion] [RFC] complex functions in npymath

2009-10-30 Thread David Cournapeau
Pauli Virtanen wrote: I can probably take a shot at this. Cool. Correct me if I'm wrong, but I think the problem is that for typedef struct foo foo_t; foo_t bar(); You're right, I was thinking about alignment issues myself - that's why I mentioned npy_complex and

Re: [Numpy-discussion] [RFC] complex functions in npymath

2009-10-30 Thread David Cournapeau
Pauli Virtanen wrote: I'd like to write code like this: coshf(a) + sinhf(b) and not like this: npy_coshf(a) + npy_sinhf(b) Using npy_ prefix was a consciously designed feature :) I would prefer avoid doing this, as it may cause trouble: sometimes, even if the foo function

Re: [Numpy-discussion] [RFC] new function for floating point comparison

2009-10-30 Thread David Cournapeau
Robert Kern wrote: That sounds good. Another worthwhile addition would be nextafter(). http://www.opengroup.org/onlinepubs/95399/functions/nextafter.html Ah, I did not know about this one. I have implemented it and committed it. One issue is that it will cause failures on platforms

[Numpy-discussion] type 'numpy.int64' unhashable

2009-10-30 Thread Sebastian Haase
Hi, I get this error: set(chainsA[0,:,0]) TypeError: unhashable type: 'numpy.ndarray' list(chainsA[0,:,0]) [2636, 2590, 2619, 2590] list(chainsA[0,:,0])[0] 2636 type(_) type 'numpy.int64' I understand where this error comes from, however what I was trying to do seems to intuitive that I would

Re: [Numpy-discussion] type 'numpy.int64' unhashable

2009-10-30 Thread David Cournapeau
On Fri, Oct 30, 2009 at 8:04 PM, Sebastian Haase seb.ha...@gmail.com wrote: I understand where this error comes from, however what I was trying to do seems to intuitive that I would like to ask for suggestions: What should I do if the number 2636 becomes unhashable ? In your example, that's

Re: [Numpy-discussion] type 'numpy.int64' unhashable

2009-10-30 Thread Gael Varoquaux
On Fri, Oct 30, 2009 at 08:21:16PM +0900, David Cournapeau wrote: On Fri, Oct 30, 2009 at 8:04 PM, Sebastian Haase seb.ha...@gmail.com wrote: I understand where this error comes from, however what I was trying to do seems to intuitive that I would like to ask for suggestions: What should I

Re: [Numpy-discussion] type 'numpy.int64' unhashable

2009-10-30 Thread James Bergstra
On Fri, Oct 30, 2009 at 7:23 AM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Fri, Oct 30, 2009 at 08:21:16PM +0900, David Cournapeau wrote: On Fri, Oct 30, 2009 at 8:04 PM, Sebastian Haase seb.ha...@gmail.com wrote: I understand where this error comes from, however what I was

[Numpy-discussion] Designing a new storage format for numpy recarrays

2009-10-30 Thread Stephen Simmons
Hi, Is anyone working on alternative storage options for numpy arrays, and specifically recarrays? My main application involves processing series of large recarrays (say 1000 recarrays, each with 5M rows having 50 fields). Existing options meet some but not all of my requirements.

Re: [Numpy-discussion] Designing a new storage format for numpy recarrays

2009-10-30 Thread Dag Sverre Seljebotn
Dag Sverre Seljebotn: Hi, Is anyone working on alternative storage options for numpy arrays, and specifically recarrays? My main application involves processing series of large recarrays (say 1000 recarrays, each with 5M rows having 50 fields). Existing options meet some but not all of my

Re: [Numpy-discussion] Designing a new storage format for numpy recarrays

2009-10-30 Thread Dag Sverre Seljebotn
Stephen Simmons wrote: P.S. Maybe this will be too much work, and I'd be better off sticking with Pytables. I can't judge that, but I want to share some thoughts (rant?): - Are you ready to not only write the code, but maintain it over years to come, and work through nasty bugs, and think

Re: [Numpy-discussion] Designing a new storage format for numpy recarrays

2009-10-30 Thread Zachary Pincus
Unless I read your request or the documentation wrong, h5py already supports pulling specific fields out of compound data types: http://h5py.alfven.org/docs-1.1/guide/hl.html#id3 For compound data, you can specify multiple field names alongside the numeric slices: dset[FieldA]

Re: [Numpy-discussion] Designing a new storage format for numpy recarrays

2009-10-30 Thread Francesc Alted
A Friday 30 October 2009 14:18:05 Stephen Simmons escrigué: - Pytables (HDF using chunked storage for recarrays with LZO compression and shuffle filter) - can't extract individual field from a recarray Er... Have you tried the ``cols`` accessor?

Re: [Numpy-discussion] Designing a new storage format for numpy recarrays

2009-10-30 Thread Anne Archibald
2009/10/30 Stephen Simmons m...@stevesimmons.com: I should clarify what I meant.. Suppose I have a recarray with 50 fields and want to read just one of those fields. PyTables/HDF will read in the compressed data for chunks of complete rows, decompress the full 50 fields, and then give me

Re: [Numpy-discussion] type 'numpy.int64' unhashable

2009-10-30 Thread Robert Kern
On Fri, Oct 30, 2009 at 08:11, James Bergstra bergs...@iro.umontreal.ca wrote: On Fri, Oct 30, 2009 at 7:23 AM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Fri, Oct 30, 2009 at 08:21:16PM +0900, David Cournapeau wrote: On Fri, Oct 30, 2009 at 8:04 PM, Sebastian Haase

Re: [Numpy-discussion] [RFC] complex functions in npymath

2009-10-30 Thread Jörgen Stenarson
David Cournapeau skrev: The missing piece in complex support in npymath is mostly tests: I have tests for all the special cases (all special cases specified in C99 standard are tested), but no test for the actual 'normal' values. If you (or someone else) could add a couple of tests, that would

[Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines

2009-10-30 Thread Reckoner
Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File

Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines

2009-10-30 Thread Robert Kern
On Fri, Oct 30, 2009 at 13:13, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: %  python -c 'import

Re: [Numpy-discussion] [RFC] complex functions in npymath

2009-10-30 Thread Pauli Virtanen
pe, 2009-10-30 kello 18:57 +0900, David Cournapeau kirjoitti: [clip: struct return values] Is this a problem in practice ? If two compilers differ in this, wouldn't they have incompatible ABI ? Yep, it would be an incompatible ABI. I don't really know how common this in practice -- but there

Re: [Numpy-discussion] type 'numpy.int64' unhashable

2009-10-30 Thread Sebastian Haase
On Fri, Oct 30, 2009 at 5:44 PM, Robert Kern robert.k...@gmail.com wrote: On Fri, Oct 30, 2009 at 08:11, James Bergstra bergs...@iro.umontreal.ca wrote: On Fri, Oct 30, 2009 at 7:23 AM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Fri, Oct 30, 2009 at 08:21:16PM +0900, David

Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines

2009-10-30 Thread Reckoner
Robert Kern wrote: You can import numpy.core.multiarray on both machines? Yes. For each machine separately, you can cPickle files with numpy arrays without problems loading/dumping. The problem comes from transferring the win32 cPickle'd files to Linux 64 bit and then trying to load them.