Re: [Numpy-discussion] categorical distributions

2010-11-22 Thread David Warde-Farley
On 2010-11-22, at 2:51 AM, Hagen Fürstenau wrote: but this is bound to be inefficient as soon as the vector of probabilities gets large, especially if you want to draw multiple samples. Have I overlooked something or should this be added? I think you misunderstand the point of multinomial

Re: [Numpy-discussion] categorical distributions

2010-11-22 Thread Hagen Fürstenau
ISTM that this elementary functionality deserves an implementation that's as fast as it can be. To substantiate this, I just wrote a simple implementation of categorical in numpy/random/mtrand.pyx and it's more than 8x faster than your version for multiple samples of the same distribution and

Re: [Numpy-discussion] categorical distributions

2010-11-22 Thread josef . pktd
On Mon, Nov 22, 2010 at 6:05 AM, Hagen Fürstenau ha...@zhuliguan.net wrote: ISTM that this elementary functionality deserves an implementation that's as fast as it can be. To substantiate this, I just wrote a simple implementation of categorical in numpy/random/mtrand.pyx and it's more than

Re: [Numpy-discussion] Slow element-by-element access?

2010-11-22 Thread Charles R Harris
On Mon, Nov 22, 2010 at 2:30 AM, Pauli Virtanen p...@iki.fi wrote: Sun, 21 Nov 2010 23:26:37 -0700, Charles R Harris wrote: [clip] Yes, indexing is known to be slow, although I don't recall the precise reason for that. Something to do with way integers are handled or some such. There was

Re: [Numpy-discussion] Does np.std() make two passes through the data?

2010-11-22 Thread Keith Goodman
On Sun, Nov 21, 2010 at 5:56 PM, Robert Kern robert.k...@gmail.com wrote: On Sun, Nov 21, 2010 at 19:49, Keith Goodman kwgood...@gmail.com wrote: But this sample gives a difference: a = np.random.rand(100) a.var()   0.080232196646619805 var(a)   0.080232196646619791 As you know, I'm

Re: [Numpy-discussion] Does np.std() make two passes through the data?

2010-11-22 Thread Benjamin Root
On Mon, Nov 22, 2010 at 11:03 AM, Keith Goodman kwgood...@gmail.com wrote: On Sun, Nov 21, 2010 at 5:56 PM, Robert Kern robert.k...@gmail.com wrote: On Sun, Nov 21, 2010 at 19:49, Keith Goodman kwgood...@gmail.com wrote: But this sample gives a difference: a = np.random.rand(100)

Re: [Numpy-discussion] Does np.std() make two passes through the data?

2010-11-22 Thread josef . pktd
On Mon, Nov 22, 2010 at 12:07 PM, Benjamin Root ben.r...@ou.edu wrote: On Mon, Nov 22, 2010 at 11:03 AM, Keith Goodman kwgood...@gmail.com wrote: On Sun, Nov 21, 2010 at 5:56 PM, Robert Kern robert.k...@gmail.com wrote: On Sun, Nov 21, 2010 at 19:49, Keith Goodman kwgood...@gmail.com

Re: [Numpy-discussion] Does np.std() make two passes through the data?

2010-11-22 Thread Keith Goodman
On Mon, Nov 22, 2010 at 9:13 AM, josef.p...@gmail.com wrote: Two pass would provide precision that we would expect in numpy, but I don't know if anyone ever tested the NIST problems for basic statistics. Here are the results for their most difficult dataset. But I guess running one test

Re: [Numpy-discussion] Does np.std() make two passes through the data?

2010-11-22 Thread Keith Goodman
On Mon, Nov 22, 2010 at 10:51 AM, josef.p...@gmail.com wrote: On Mon, Nov 22, 2010 at 1:39 PM, Keith Goodman kwgood...@gmail.com wrote: On Mon, Nov 22, 2010 at 10:32 AM,  josef.p...@gmail.com wrote: On Mon, Nov 22, 2010 at 1:26 PM, Keith Goodman kwgood...@gmail.com wrote: On Mon, Nov 22, 2010

Re: [Numpy-discussion] Does np.std() make two passes through the data?

2010-11-22 Thread josef . pktd
On Mon, Nov 22, 2010 at 1:51 PM, josef.p...@gmail.com wrote: On Mon, Nov 22, 2010 at 1:39 PM, Keith Goodman kwgood...@gmail.com wrote: On Mon, Nov 22, 2010 at 10:32 AM,  josef.p...@gmail.com wrote: On Mon, Nov 22, 2010 at 1:26 PM, Keith Goodman kwgood...@gmail.com wrote: On Mon, Nov 22, 2010

Re: [Numpy-discussion] Does np.std() make two passes through the data?

2010-11-22 Thread Keith Goodman
On Mon, Nov 22, 2010 at 11:00 AM, josef.p...@gmail.com wrote: I don't think that works for complex numbers. (statsmodels has now a preference that calculations work also for complex numbers) I'm only supporting int32, int64, float64 for now. Getting the other ints and floats should be easy.

Re: [Numpy-discussion] Does np.std() make two passes through the data?

2010-11-22 Thread josef . pktd
On Mon, Nov 22, 2010 at 1:59 PM, Keith Goodman kwgood...@gmail.com wrote: On Mon, Nov 22, 2010 at 10:51 AM,  josef.p...@gmail.com wrote: On Mon, Nov 22, 2010 at 1:39 PM, Keith Goodman kwgood...@gmail.com wrote: On Mon, Nov 22, 2010 at 10:32 AM,  josef.p...@gmail.com wrote: On Mon, Nov 22, 2010

Re: [Numpy-discussion] indexing question

2010-11-22 Thread Christopher Barker
On 11/21/10 11:37 AM, Ernest Adrogué wrote: so you want t[:,x,y] I tried that, but it's not the same: In [307]: t[[0,1],x,y] Out[307]: array([1, 7]) In [308]: t[:,x,y] Out[308]: array([[1, 3], [5, 7]]) what is your t? Here's my example, which I think matches what you asked

Re: [Numpy-discussion] Does np.std() make two passes through the data?

2010-11-22 Thread josef . pktd
On Mon, Nov 22, 2010 at 2:04 PM, Keith Goodman kwgood...@gmail.com wrote: On Mon, Nov 22, 2010 at 11:00 AM,  josef.p...@gmail.com wrote: I don't think that works for complex numbers. (statsmodels has now a preference that calculations work also for complex numbers) I'm only supporting

Re: [Numpy-discussion] indexing question

2010-11-22 Thread John Salvatier
I didn't realize the x's and y's were varying the first time around. There's probably a way to omit it, but I think the conceptually simplest way is probably what you had to begin with. Build an index by saying i = numpy.arange(0, t.shape[0]) then you can do t[i, x,y] On Mon, Nov 22, 2010 at

Re: [Numpy-discussion] ANN: NumPy 1.5.1

2010-11-22 Thread Christopher Barker
On 11/20/10 11:04 PM, Ralf Gommers wrote: I am pleased to announce the availability of NumPy 1.5.1. Binaries, sources and release notes can be found at https://sourceforge.net/projects/numpy/files/. Thank you to everyone who contributed to this release. Yes, thanks so much -- in particular

Re: [Numpy-discussion] ANN: NumPy 1.5.1

2010-11-22 Thread Matthew Brett
Hi, On Mon, Nov 22, 2010 at 11:35 AM, Christopher Barker chris.bar...@noaa.gov wrote: On 11/20/10 11:04 PM, Ralf Gommers wrote: I am pleased to announce the availability of NumPy 1.5.1. Binaries, sources and release notes can be found at https://sourceforge.net/projects/numpy/files/. Thank

[Numpy-discussion] N dimensional dichotomy optimization

2010-11-22 Thread Gael Varoquaux
Hi list, does anybody have, or knows where I can find some N dimensional dichotomy optimization code in Python (BSD licensed, or equivalent)? Worst case, it does not look too bad to code, but I am interested by any advice. I haven't done my reading yet, and I don't know how ill-posed a problem

Re: [Numpy-discussion] indexing question

2010-11-22 Thread Ernest Adrogué
22/11/10 @ 11:08 (-0800), thus spake Christopher Barker: On 11/21/10 11:37 AM, Ernest Adrogué wrote: so you want t[:,x,y] I tried that, but it's not the same: In [307]: t[[0,1],x,y] Out[307]: array([1, 7]) In [308]: t[:,x,y] Out[308]: array([[1, 3], [5, 7]]) what is

Re: [Numpy-discussion] indexing question

2010-11-22 Thread Robert Kern
2010/11/21 Ernest Adrogué eadro...@gmx.net: Hi, Suppose an array of shape (N,2,2), that is N arrays of shape (2,2). I want to select an element (x,y) from each one of the subarrays, so I get a 1-dimensional array of length N. For instance: In [228]: t=np.arange(8).reshape(2,2,2) In

Re: [Numpy-discussion] indexing question

2010-11-22 Thread Ernest Adrogué
22/11/10 @ 11:20 (-0800), thus spake John Salvatier: I didn't realize the x's and y's were varying the first time around. There's probably a way to omit it, but I think the conceptually simplest way is probably what you had to begin with. Build an index by saying i = numpy.arange(0,

Re: [Numpy-discussion] N dimensional dichotomy optimization

2010-11-22 Thread Matthieu Brucher
2010/11/22 Gael Varoquaux gael.varoqu...@normalesup.org: Hi list, Hi ;) does anybody have, or knows where I can find some N dimensional dichotomy optimization code in Python (BSD licensed, or equivalent)? I don't know any code, but it should be too difficult by bgoing through a KdTree.

Re: [Numpy-discussion] indexing question

2010-11-22 Thread Ernest Adrogué
22/11/10 @ 14:04 (-0600), thus spake Robert Kern: This way, I get the elements (0,1) and (1,1) which is what I wanted. The question is: is it possible to omit the [0,1] in the index? No, but you can write generic code for it: t[np.arange(t.shape[0]), x, y] Thank you. This is what I

Re: [Numpy-discussion] indexing question

2010-11-22 Thread John Salvatier
I think that the only speedup you will get is defining an index only once and reusing it. 2010/11/22 Ernest Adrogué eadro...@gmx.net: 22/11/10 @ 14:04 (-0600), thus spake Robert Kern: This way, I get the elements (0,1) and (1,1) which is what I wanted. The question is: is it possible to omit

Re: [Numpy-discussion] N dimensional dichotomy optimization

2010-11-22 Thread Gael Varoquaux
On Mon, Nov 22, 2010 at 09:12:45PM +0100, Matthieu Brucher wrote: Hi ;) Hi bro does anybody have, or knows where I can find some N dimensional dichotomy optimization code in Python (BSD licensed, or equivalent)? I don't know any code, but it should be too difficult by bgoing through a

Re: [Numpy-discussion] N dimensional dichotomy optimization

2010-11-22 Thread Matthieu Brucher
2010/11/22 Gael Varoquaux gael.varoqu...@normalesup.org: On Mon, Nov 22, 2010 at 09:12:45PM +0100, Matthieu Brucher wrote: Hi ;) Hi bro does anybody have, or knows where I can find some N dimensional dichotomy optimization code in Python (BSD licensed, or equivalent)? I don't know any

Re: [Numpy-discussion] N dimensional dichotomy optimization

2010-11-22 Thread Gael Varoquaux
On Mon, Nov 22, 2010 at 11:12:26PM +0100, Matthieu Brucher wrote: It seems that a simplex is what you need. Ha! I am learning new fancy words. Now I can start looking clever. I realize that maybe I should rephrase my question to try and draw more out of the common wealth of knowledge on

Re: [Numpy-discussion] N dimensional dichotomy optimization

2010-11-22 Thread Matthieu Brucher
2010/11/22 Gael Varoquaux gael.varoqu...@normalesup.org: On Mon, Nov 22, 2010 at 11:12:26PM +0100, Matthieu Brucher wrote: It seems that a simplex is what you need. Ha! I am learning new fancy words. Now I can start looking clever. I realize that maybe I should rephrase my question to try

Re: [Numpy-discussion] N dimensional dichotomy optimization

2010-11-22 Thread Gael Varoquaux
On Mon, Nov 22, 2010 at 11:12:26PM +0100, Matthieu Brucher wrote: It seems that a simplex is what you need. It uses the barycenter (more or less) to find a new point in the simplex. And it works well only in convex functions (but in fact almost all functions have an issue with this :D) One

Re: [Numpy-discussion] N dimensional dichotomy optimization

2010-11-22 Thread josef . pktd
On Mon, Nov 22, 2010 at 5:27 PM, Matthieu Brucher matthieu.bruc...@gmail.com wrote: 2010/11/22 Gael Varoquaux gael.varoqu...@normalesup.org: On Mon, Nov 22, 2010 at 11:12:26PM +0100, Matthieu Brucher wrote: It seems that a simplex is what you need. Ha! I am learning new fancy words. Now I can

Re: [Numpy-discussion] Matlab IO Warning in mio5.py

2010-11-22 Thread Alexander Kain
It's not an error but a harmless (although confusing) warning message. You should be able to filter it by adding the following to scipy/__init__.py: import warnings warnings.filterwarnings(action='ignore', message='.*__builtin__.file size changed.*') Can you check if that works for you?

Re: [Numpy-discussion] N dimensional dichotomy optimization

2010-11-22 Thread Matthieu Brucher
2010/11/22 Gael Varoquaux gael.varoqu...@normalesup.org: On Mon, Nov 22, 2010 at 11:12:26PM +0100, Matthieu Brucher wrote: It seems that a simplex is what you need. It uses the barycenter (more or less) to find a new point in the simplex. And it works well only in convex functions (but in fact

[Numpy-discussion] numpy.test() errors

2010-11-22 Thread Nils Wagner
Hi all, There are some new test errors == ERROR: Test with missing and filling values -- Traceback (most recent call last): File