[Numpy-discussion] interleaved indexing

2008-07-18 Thread Amir
A very beginner question about indexing: let x be an array where n = len(x). I would like to create a view y of x such that: y[i] = x[i:i+m,...] for each i and a fixed m n so I can do things like numpy.cov(y). With n large, allocating y is a problem for me. Currently, I either do for loops in

[Numpy-discussion] Type checking of arrays containing strings

2008-07-18 Thread Arnar Flatberg
Hi I need to check if my array (a) is of type `string`. That is, I dont know the number of characters beforehand, so I cant do a.dtype == '|S*' (* = (max) number of characters) Looking at my options, I see either a.dtype.kind == 'S' or a.dtype.type == np.string_, might be ok. Are these any of

Re: [Numpy-discussion] interleaved indexing

2008-07-18 Thread Stéfan van der Walt
Hi Amir 2008/7/18 Amir [EMAIL PROTECTED]: A very beginner question about indexing: let x be an array where n = len(x). I would like to create a view y of x such that: y[i] = x[i:i+m,...] for each i and a fixed m n so I can do things like numpy.cov(y). With n large, allocating y is a

Re: [Numpy-discussion] Type checking of arrays containing strings

2008-07-18 Thread Stéfan van der Walt
2008/7/18 Arnar Flatberg [EMAIL PROTECTED]: I need to check if my array (a) is of type `string`. That is, I dont know the number of characters beforehand, so I cant do a.dtype == '|S*' (* = (max) number of characters) Looking at my options, I see either a.dtype.kind == 'S' or a.dtype.type ==

Re: [Numpy-discussion] Type checking of arrays containing strings

2008-07-18 Thread Arnar Flatberg
On Fri, Jul 18, 2008 at 11:40 AM, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/7/18 Arnar Flatberg [EMAIL PROTECTED]: I need to check if my array (a) is of type `string`. That is, I dont know the number of characters beforehand, so I cant do a.dtype == '|S*' (* = (max) number of

Re: [Numpy-discussion] Type checking of arrays containing strings

2008-07-18 Thread Arnar Flatberg
On Fri, Jul 18, 2008 at 12:38 PM, Arnar Flatberg [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 11:40 AM, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/7/18 Arnar Flatberg [EMAIL PROTECTED]: I need to check if my array (a) is of type `string`. That is, I dont know the number of

Re: [Numpy-discussion] Type checking of arrays containing strings

2008-07-18 Thread Stéfan van der Walt
2008/7/18 Arnar Flatberg [EMAIL PROTECTED]: ... and instead of just complaining, I can do something about it :-) Could you add permissions to me for the documentation editor, username: That's the spirit! You are now added. But I'm worried: I had a bet with Joe that we won't get more than 30

[Numpy-discussion] chararray __mod__ behavior

2008-07-18 Thread Alan McIntyre
This seems odd to me: A=np.array([['%.3f','%d'],['%s','%r']]).view(np.chararray) A % np.array([[1,2],[3,4]]) Traceback (most recent call last): File stdin, line 1, in module File /opt/local/lib/python2.5/site-packages/numpy/core/defchararray.py, line 126, in __mod__ newarr[:] = res

Re: [Numpy-discussion] chararray __mod__ behavior

2008-07-18 Thread Stéfan van der Walt
2008/7/18 Alan McIntyre [EMAIL PROTECTED]: This seems odd to me: A=np.array([['%.3f','%d'],['%s','%r']]).view(np.chararray) A % np.array([[1,2],[3,4]]) Traceback (most recent call last): File stdin, line 1, in module File /opt/local/lib/python2.5/site-packages/numpy/core/defchararray.py,

Re: [Numpy-discussion] RFC: A (second) proposal for implementing some date/time types in NumPy

2008-07-18 Thread Ivan Vilata i Balaguer
Francesc Alted (el 2008-07-16 a les 18:44:36 +0200) va dir:: After tons of excellent feedback received for our first proposal about the date/time types in NumPy Ivan and me have had another brainstorming session and ended with a new proposal for your consideration. After re-reading the

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Charles R Harris
On Fri, Jul 18, 2008 at 5:15 AM, Michael Abbott [EMAIL PROTECTED] wrote: I'm afraid this is going to be a long one, and I don't see any good way to cut down the quoted text either... Charles, I'm going to plea with you to read what I've just written and think about it. I'm trying to make

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Charles R Harris
On Fri, Jul 18, 2008 at 12:03 PM, Charles R Harris [EMAIL PROTECTED] wrote: snip Simpler test case: import sys, gc import numpy as np def main() : t = np.dtype(np.float32) print sys.getrefcount(t) for i in range(100) : np.float32() gc.collect() print

[Numpy-discussion] building a better OSX install for 1.1.1

2008-07-18 Thread Christopher Burns
Sorry Russell, I was a bit brief before. Installer.app checks for a system requirement when the user installs numpy. I build numpy using bdist_mpkg against the python.org version of python (MacPython). If a user tries to install numpy and they _do not_ have this version of python installed,

[Numpy-discussion] integer array creation oddity

2008-07-18 Thread Suchindra Sandhu
Hi, Can someone please explain to me this oddity? In [1]: import numpy as n In [8]: a = n.array((1,2,3), 'i') In [9]: type(a[0]) Out[9]: type 'numpy.int32' In [10]: type(a[0]) == n.int32 Out[10]: False When I create an array with 'int', 'int32' etc it works fine What is the type of 'i' and

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Charles R Harris
On Fri, Jul 18, 2008 at 2:41 PM, Charles R Harris [EMAIL PROTECTED] wrote: snip There are actually two bugs here which is confusing things. Bug 1) Deleting a numpy scalar doesn't decrement the descr reference count. Bug 2) PyArray_Return decrements the reference to arr, which in turn correctly

[Numpy-discussion] numpy.loadtext() fails with dtype + usecols

2008-07-18 Thread Ryan May
Hi, I was trying to use loadtxt() today to read in some text data, and I had a problem when I specified a dtype that only contained as many elements as in columns in usecols. The example below shows the problem: import numpy as np import StringIO data = '''STID RELH TAIR JOE 70.1 25.3 BOB

[Numpy-discussion] f2py - how to use .pyf files?

2008-07-18 Thread Fernando Perez
Howdy, again f2py... Since I can't seem to figure out how to pass the --fcompiler option to f2py via setup.py/distutils, I decided to just do things for now via a plain makefile. But I'm struggling here too. The problem is this: the call f2py -c --fcompiler=gfortran -m text2 Text2.f90 works

Re: [Numpy-discussion] numpy.loadtext() fails with dtype + usecols

2008-07-18 Thread Charles R Harris
On Fri, Jul 18, 2008 at 4:16 PM, Ryan May [EMAIL PROTECTED] wrote: Hi, I was trying to use loadtxt() today to read in some text data, and I had a problem when I specified a dtype that only contained as many elements as in columns in usecols. The example below shows the problem: import

Re: [Numpy-discussion] f2py - how to use .pyf files?

2008-07-18 Thread Robert Kern
On Fri, Jul 18, 2008 at 17:39, Fernando Perez [EMAIL PROTECTED] wrote: Howdy, again f2py... Since I can't seem to figure out how to pass the --fcompiler option to f2py via setup.py/distutils, I decided to just do things for now via a plain makefile. But I'm struggling here too. The problem

Re: [Numpy-discussion] f2py - how to use .pyf files?

2008-07-18 Thread Robert Kern
On Fri, Jul 18, 2008 at 18:19, Fernando Perez [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 3:53 PM, Robert Kern [EMAIL PROTECTED] wrote: You still need -m text2, I believe. Right, thanks. But it still doesn't quite work. Consider a makefile with lib: seglib.so seglib.so: Text2.f90

Re: [Numpy-discussion] f2py - how to use .pyf files?

2008-07-18 Thread Fernando Perez
On Fri, Jul 18, 2008 at 4:26 PM, Robert Kern [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 18:19, Fernando Perez [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 3:53 PM, Robert Kern [EMAIL PROTECTED] wrote: You still need -m text2, I believe. Right, thanks. But it still doesn't quite

Re: [Numpy-discussion] [matplotlib-devel] Matplotlib and latest numpy

2008-07-18 Thread Gael Varoquaux
On Sat, Jul 19, 2008 at 01:25:51AM +0200, Gael Varoquaux wrote: Am I wrong, or does matploib not build with current numpy svn? OK, Fernando told me that matplotlib builds fine with latest numpy on his box so Ienquired a bit more. The problem is that the build of matplotlib tries to include a

Re: [Numpy-discussion] [matplotlib-devel] Matplotlib and latest numpy

2008-07-18 Thread Robert Kern
On Fri, Jul 18, 2008 at 18:40, Gael Varoquaux [EMAIL PROTECTED] wrote: On Sat, Jul 19, 2008 at 01:25:51AM +0200, Gael Varoquaux wrote: Am I wrong, or does matploib not build with current numpy svn? OK, Fernando told me that matplotlib builds fine with latest numpy on his box so Ienquired a

Re: [Numpy-discussion] [matplotlib-devel] Matplotlib and latest numpy

2008-07-18 Thread Charles R Harris
On Fri, Jul 18, 2008 at 5:44 PM, Robert Kern [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 18:40, Gael Varoquaux [EMAIL PROTECTED] wrote: On Sat, Jul 19, 2008 at 01:25:51AM +0200, Gael Varoquaux wrote: Am I wrong, or does matploib not build with current numpy svn? OK, Fernando told

Re: [Numpy-discussion] f2py - how to use .pyf files?

2008-07-18 Thread Robert Kern
On Fri, Jul 18, 2008 at 18:35, Fernando Perez [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 4:26 PM, Robert Kern [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 18:19, Fernando Perez [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 3:53 PM, Robert Kern [EMAIL PROTECTED] wrote: You still

Re: [Numpy-discussion] [matplotlib-devel] Matplotlib and latest numpy

2008-07-18 Thread Gael Varoquaux
On Fri, Jul 18, 2008 at 06:44:21PM -0500, Robert Kern wrote: It's not a setuptools issue at all. build_ext --inplace just doesn't install header files. Does that mean that python setup.py develop should be banned for numpy? If so I consider it a setuptools issue: one more caveat to learn about

Re: [Numpy-discussion] [matplotlib-devel] Matplotlib and latest numpy

2008-07-18 Thread Robert Kern
On Fri, Jul 18, 2008 at 18:49, Gael Varoquaux [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 06:44:21PM -0500, Robert Kern wrote: It's not a setuptools issue at all. build_ext --inplace just doesn't install header files. Does that mean that python setup.py develop should be banned for

Re: [Numpy-discussion] [matplotlib-devel] Matplotlib and latest numpy

2008-07-18 Thread Charles R Harris
On Fri, Jul 18, 2008 at 5:53 PM, Robert Kern [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 18:49, Gael Varoquaux [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 06:44:21PM -0500, Robert Kern wrote: It's not a setuptools issue at all. build_ext --inplace just doesn't install header

Re: [Numpy-discussion] f2py - how to use .pyf files?

2008-07-18 Thread Fernando Perez
On Fri, Jul 18, 2008 at 4:47 PM, Robert Kern [EMAIL PROTECTED] wrote: For publicly distributed packages, it does go against the grain. Never hard-code the compiler! Use a setup.cfg file, instead. Quite all right. But this was for in-house code where a group has agreed to all use the same

Re: [Numpy-discussion] [matplotlib-devel] Matplotlib and latest numpy

2008-07-18 Thread Robert Kern
On Fri, Jul 18, 2008 at 19:02, Charles R Harris [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 5:53 PM, Robert Kern [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 18:49, Gael Varoquaux [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 06:44:21PM -0500, Robert Kern wrote: It's not a

Re: [Numpy-discussion] f2py - how to use .pyf files?

2008-07-18 Thread Robert Kern
On Fri, Jul 18, 2008 at 19:04, Fernando Perez [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 4:47 PM, Robert Kern [EMAIL PROTECTED] wrote: For publicly distributed packages, it does go against the grain. Never hard-code the compiler! Use a setup.cfg file, instead. Quite all right. But

Re: [Numpy-discussion] f2py - how to use .pyf files?

2008-07-18 Thread Fernando Perez
On Fri, Jul 18, 2008 at 5:15 PM, Robert Kern [EMAIL PROTECTED] wrote: Probably. What other options would you need, though? If everything is accessible from setup.cfg, I'd rather just remove the argument. I'd be OK with that too, and simply telling people to ship a companion setup.cfg. It's

Re: [Numpy-discussion] [matplotlib-devel] Matplotlib and latest numpy

2008-07-18 Thread Robert Kern
On Fri, Jul 18, 2008 at 19:07, Robert Kern [EMAIL PROTECTED] wrote: The code generation logic in numpy does not know anything about build_ext --inplace (which is a distutils feature that develop invokes). That's it. This problem has always existed for all versions of numpy whether you use

Re: [Numpy-discussion] [matplotlib-devel] Matplotlib and latest numpy

2008-07-18 Thread Gael Varoquaux
On Fri, Jul 18, 2008 at 08:14:00PM -0500, Robert Kern wrote: The code generation logic in numpy does not know anything about build_ext --inplace (which is a distutils feature that develop invokes). That's it. This problem has always existed for all versions of numpy whether you use

Re: [Numpy-discussion] [matplotlib-devel] Matplotlib and latest numpy

2008-07-18 Thread Charles R Harris
On Fri, Jul 18, 2008 at 7:14 PM, Robert Kern [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 19:07, Robert Kern [EMAIL PROTECTED] wrote: The code generation logic in numpy does not know anything about build_ext --inplace (which is a distutils feature that develop invokes). That's it.

Re: [Numpy-discussion] f2py - how to use .pyf files?

2008-07-18 Thread Fernando Perez
On Fri, Jul 18, 2008 at 5:20 PM, Fernando Perez [EMAIL PROTECTED] wrote: On Fri, Jul 18, 2008 at 5:15 PM, Robert Kern [EMAIL PROTECTED] wrote: Probably. What other options would you need, though? If everything is accessible from setup.cfg, I'd rather just remove the argument. I just

Re: [Numpy-discussion] Another reference count leak: ticket #848

2008-07-18 Thread Travis E. Oliphant
Looking at the uses of PyArray_FromAny I can see the motivation for this design: core/include/numpy/ndarrayobject.h has a lot of calls which take a value returned by PyArray_DescrFromType as argument. This has prompted me to take a trawl through the code to see what else is going on, and

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Travis E. Oliphant
Michael Abbott wrote: Only half of my patch for this bug has gone into trunk, and without the rest of my patch there remains a leak. Thanks for your work Michael. I've been so grateful to have you and Chuck and others looking carefully at the code to fix its problems. In this

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Travis E. Oliphant
Charles R Harris wrote: The reference leak seems specific to the float32 and complex64 types called with default arguments. In [1]: import sys, gc In [2]: t = float32 In [3]: sys.getrefcount(dtype(t)) Out[3]: 4 In [4]: for i in range(10) : t(); ...: In [5]:

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Charles R Harris
On Fri, Jul 18, 2008 at 9:15 PM, Travis E. Oliphant [EMAIL PROTECTED] wrote: Michael Abbott wrote: Only half of my patch for this bug has gone into trunk, and without the rest of my patch there remains a leak. Thanks for your work Michael. I've been so grateful to have you and Chuck

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Travis E. Oliphant
Charles R Harris wrote: On Fri, Jul 18, 2008 at 9:15 PM, Travis E. Oliphant [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Michael Abbott wrote: Only half of my patch for this bug has gone into trunk, and without the rest of my patch there remains a leak.

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Charles R Harris
On Fri, Jul 18, 2008 at 9:30 PM, Travis E. Oliphant [EMAIL PROTECTED] wrote: Charles R Harris wrote: The reference leak seems specific to the float32 and complex64 types called with default arguments. In [1]: import sys, gc In [2]: t = float32 In [3]:

[Numpy-discussion] f2py - a recap

2008-07-18 Thread Fernando Perez
Howdy, today's exercise with f2py left some lessons learned, mostly thanks to Robert's excellent help, for which I'm grateful. I'd like to recap here what we have, mostly to decide what changes (if any) should go into numpy to make the experience less interesting for future users: - Remove the

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Travis E. Oliphant
Charles R Harris wrote: On Fri, Jul 18, 2008 at 9:15 PM, Travis E. Oliphant [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Michael Abbott wrote: Only half of my patch for this bug has gone into trunk, and without the rest of my patch there remains a leak.

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Travis E. Oliphant
Charles R Harris wrote: On Fri, Jul 18, 2008 at 9:15 PM, Travis E. Oliphant [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Michael Abbott wrote: Only half of my patch for this bug has gone into trunk, and without the rest of my patch there remains a leak.

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Travis E. Oliphant
I've attached a test script. Thank you! It looks like with that added DECREF, the reference count leak is gone.While it was a minor issue (it should be noted that reference counting errors on the built-in data-types won't cause issues), it is nice to clean these things up when we can.

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Gael Varoquaux
On Fri, Jul 18, 2008 at 11:35:50PM -0500, Travis E. Oliphant wrote: I've attached a test script. Thank you! It looks like with that added DECREF, the reference count leak is gone.While it was a minor issue (it should be noted that reference counting errors on the built-in data-types

Re: [Numpy-discussion] Ticket review: #848, leak in PyArray_DescrFromType

2008-07-18 Thread Charles R Harris
On Fri, Jul 18, 2008 at 10:04 PM, Travis E. Oliphant [EMAIL PROTECTED] wrote: Charles R Harris wrote: On Fri, Jul 18, 2008 at 9:15 PM, Travis E. Oliphant [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Michael Abbott wrote: Only half of my patch for this bug has gone