Re: [Numpy-discussion] Bug in genfromtxt with usecols and converters

2014-08-27 Thread Derek Homeier
On 26 Aug 2014, at 09:05 pm, Adrian Altenhoff adrian.altenh...@inf.ethz.ch wrote: But you are right that the problem with using the first_values, which should of course be valid, somehow stems from the use of usecols, it seems that in that loop for (i, conv) in user_converters.items():

[Numpy-discussion] Convert 3d NumPy array into 2d

2014-08-27 Thread phinn stuart
Hi everyone, how can I convert (1L, 480L, 1440L) shaped numpy array into (480L, 1440L)? Thanks in the advance. phinn ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Convert 3d NumPy array into 2d

2014-08-27 Thread Wagner Sebastian
Hi, Our short example-data: np.arange(10).reshape(1,5,2) array([[[0, 1], [2, 3], [4, 5], [6, 7], [8, 9]]]) Shape is (1,5,2) Two possibilies: data.reshape(5,2) array([[0, 1], [2, 3], [4, 5], [6, 7], [8, 9]]) Or just: data[0]

Re: [Numpy-discussion] Convert 3d NumPy array into 2d

2014-08-27 Thread Benjamin Root
There is also np.squeeze(), which will eliminate any singleton dimensions (but I personally hate using it because it can accidentally squeeze out dimensions that you didn't intend to squeeze when you have arbitrary input data). Ben Root On Wed, Aug 27, 2014 at 11:12 AM, Wagner Sebastian

Re: [Numpy-discussion] Convert 3d NumPy array into 2d

2014-08-27 Thread Julian Taylor
On 27.08.2014 17:08, phinn stuart wrote: Hi everyone, how can I convert (1L, 480L, 1440L) shaped numpy array into (480L, 1440L)? Thanks in the advance. np.squeeze removes empty dimensions: In [2]: np.squeeze(np.ones((1,23,232))).shape Out[2]: (23, 232)

[Numpy-discussion] Should concatenate broadcast shapes?

2014-08-27 Thread Jaime Fernández del Río
After reading this stackoverflow question: http://stackoverflow.com/questions/25530223/append-a-list-at-the-end-of-each-row-of-2d-array I was reminded that the `np.concatenate` family of functions do not broadcast the shapes of their inputs: import numpy as np a = np.arange(6).reshape(3, 2)

Re: [Numpy-discussion] Should concatenate broadcast shapes?

2014-08-27 Thread Robert Kern
On Wed, Aug 27, 2014 at 5:44 PM, Jaime Fernández del Río jaime.f...@gmail.com wrote: After reading this stackoverflow question: http://stackoverflow.com/questions/25530223/append-a-list-at-the-end-of-each-row-of-2d-array I was reminded that the `np.concatenate` family of functions do not

[Numpy-discussion] Does a `mergesorted` function make sense?

2014-08-27 Thread Jaime Fernández del Río
A request was open in github to add a `merge` function to numpy that would merge two sorted 1d arrays into a single sorted 1d array. I have been playing around with that idea for a while, and have a branch in my numpy fork that adds a `mergesorted` function to `numpy.lib`:

[Numpy-discussion] ANN: NumPy 1.9.0 release candidate 1 available

2014-08-27 Thread Julian Taylor
Hello, Almost punctually for EuroScipy we have finally managed to release the first release candidate of NumPy 1.9. We intend to only fix bugs until the final release which we plan to do in the next 1-2 weeks. In this release numerous performance improvements have been added, most significantly

Re: [Numpy-discussion] Should concatenate broadcast shapes?

2014-08-27 Thread Jaime Fernández del Río
On Wed, Aug 27, 2014 at 10:01 AM, Robert Kern robert.k...@gmail.com wrote: On Wed, Aug 27, 2014 at 5:44 PM, Jaime Fernández del Río jaime.f...@gmail.com wrote: After reading this stackoverflow question:

Re: [Numpy-discussion] Does a `mergesorted` function make sense?

2014-08-27 Thread Eelco Hoogendoorn
It wouldn't hurt to have this function, but my intuition is that its use will be minimal. If you are already working with sorted arrays, you already have a flop cost on that order of magnitude, and the optimized merge saves you a factor two at the very most. Using numpy means you are sacrificing

Re: [Numpy-discussion] Does a `mergesorted` function make sense?

2014-08-27 Thread Jaime Fernández del Río
Hi Eelco, I took a deeper look into your code a couple of weeks back. I don't think I have fully grasped what it allows completely, but I agree that some form of what you have there is highly desirable. Along the same lines, for sometime I have been thinking that the right place for a `groupby`

Re: [Numpy-discussion] Does a `mergesorted` function make sense?

2014-08-27 Thread Eelco Hoogendoorn
If I understand you correctly, the current implementation supports these operations. All reductions over groups (except for median) are performed through the corresponding ufunc (see GroupBy.reduce). This works on multidimensional arrays as well, although this broadcasting over the non-grouping

Re: [Numpy-discussion] Does a `mergesorted` function make sense?

2014-08-27 Thread Eelco Hoogendoorn
i.e, if the grouped axis is small but the other axes are not, you could write this, which avoids the python loop over the long axis that np.vectorize would otherwise perform. import numpy as np from grouping import group_by keys = np.random.randint(0,4,10) values = np.random.rand(10,2000) for k,g

Re: [Numpy-discussion] Does a `mergesorted` function make sense?

2014-08-27 Thread Jaime Fernández del Río
Yes, I was aware of that. But the point would be to provide true vectorization on those operations. The way I see it, numpy may not have to have a GroupBy implementation, but it should at least enable implementing one that is fast and efficient over any axis. On Wed, Aug 27, 2014 at 12:38 PM,

Re: [Numpy-discussion] ANN: NumPy 1.9.0 release candidate 1 available

2014-08-27 Thread Orion Poplawski
On 08/27/2014 11:07 AM, Julian Taylor wrote: Hello, Almost punctually for EuroScipy we have finally managed to release the first release candidate of NumPy 1.9. We intend to only fix bugs until the final release which we plan to do in the next 1-2 weeks. I'm seeing the following errors

Re: [Numpy-discussion] Does a `mergesorted` function make sense?

2014-08-27 Thread Daπid
On 27 August 2014 19:02, Jaime Fernández del Río jaime.f...@gmail.com wrote: Since there is at least one other person out there that likes it, is there any more interest in such a function? If yes, any comments on what the proper interface for extra output should be? Although perhaps the best

Re: [Numpy-discussion] Does a `mergesorted` function make sense?

2014-08-27 Thread Eelco Hoogendoorn
I just checked the docs on ufuncs, and it appears that's a solved problem now, since ufunc.reduceat now comes with an axis argument. Or maybe it already did when I wrote that, but I simply wasn't paying attention. Either way, the code is fully vectorized now, in both grouped and non-grouped axes.

Re: [Numpy-discussion] ANN: NumPy 1.9.0 release candidate 1 available

2014-08-27 Thread Charles R Harris
On Wed, Aug 27, 2014 at 3:52 PM, Orion Poplawski or...@cora.nwra.com wrote: On 08/27/2014 11:07 AM, Julian Taylor wrote: Hello, Almost punctually for EuroScipy we have finally managed to release the first release candidate of NumPy 1.9. We intend to only fix bugs until the final