[Numpy-discussion] dot and MKL memory
Hi, If I build Python 2.7.2 and numpy-1.9.1 and run the following script import numpy c = numpy.ones((50, 4)) mat = numpy.identity(4) r = numpy.dot(c, mat) the evaluation of the 'dot' increases the memory by app. 35 MB. If, in addition, I build numpy-1.9.1 with MKL support, and run the script, the evaluation of the 'dot' increases the memory by app. 450 MB. Is the expected? Best regards, Mads specs: Ubuntu 12.04 ifort (IFORT) 14.0.1 2013100 gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) -- +-+ | Mads Ipsen | +--+--+ | Overgaden Oven Vandet 106, 4.tv | phone: +45-29716388 | | DK-1415 København K | email: mads.ip...@gmail.com | | Denmark | map : https://goo.gl/maps/oQ6y6 | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Detect if array has been transposed
On 13/10/14 01:18, Nathaniel Smith wrote: On Mon, Oct 13, 2014 at 12:07 AM, Pauli Virtanen p...@iki.fi wrote: 12.10.2014, 22:16, Eric Firing kirjoitti: On 2014/10/12, 8:29 AM, Pauli Virtanen wrote: 12.10.2014, 20:19, Mads Ipsen kirjoitti: Is there any way for me to detect (on the Python side) that transpose() has been invoked on the matrix, and thereby only do the copy operation when it really is needed? The correct way to do this is to, either: In your C code check PyArray_IS_C_CONTIGUOUS(obj) and raise an error if it is not. In addition, on the Python side, check for `a.flags.c_contiguous` and make a copy if it is not. OR In your C code, get an handle to the array using PyArray_FromANY (or PyArray_FromOTF) with NPY_ARRAY_C_CONTIGUOUS requirement set so that it makes a copy when necessary. or let numpy handle it on the python side: foo(numpy.ascontiguousarray(a)) Yes, but the C code really should check that the input array is C-contiguous, if it only works for C-contiguous inputs. I.e. your original instructions were correct, but instead of checking a.flags.c_contiguous by hand etc. the OP should just call ascontiguousarray which takes care of that part. Hi, To everybody that answered - your help is (as always) much appreciated. Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | phone: +45-29716388 | | DK-2500 Valby| email: mads.ip...@gmail.com | | Denmark | map : www.tinyurl.com/ns52fpa | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Detect if array has been transposed
Hi,
In part of my C++ code, I often do
void foo(PyObject * matrix) {
do stuff
}
where matrix is a numpy mxn matrix created on the Python side, where
foo() eg. is invoked as
a = numpy.array([[1,2],[3,5]])
foo(a)
However, if you call transpose() on a, some care should be taken, since
numpy's internal matrix data first gets transposed on demand. In that
case I must do
a = numpy.array([[1,2],[3,5]])
a.transpose()
foo(a.copy())
to make sure the correct data of the array gets transferred to the C++
side.
Is there any way for me to detect (on the Python side) that transpose()
has been invoked on the matrix, and thereby only do the copy operation
when it really is needed? For example
if a_has_transposed_data:
foo(a.copy())
else:
foo(a)
Best regards,
Mads
--
+-+
| Mads Ipsen |
+--+--+
| Gåsebæksvej 7, 4. tv | phone: +45-29716388 |
| DK-2500 Valby| email: mads.ip...@gmail.com |
| Denmark | map : www.tinyurl.com/ns52fpa |
+--+--+
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Tracking and inspecting numpy objects
Hi, I am trying to inspect the reference count of numpy arrays generated by my application. Initially, I thought I could inspect the tracked objects using gc.get_objects(), but, with respect to numpy objects, the returned container is empty. For example: import numpy import gc data = numpy.ones(1024).reshape((32,32)) objs = [o for o in gc.get_objects() if isinstance(o, numpy.ndarray)] print objs# Prints empty list print gc.is_tracked(data) # Print False Why is this? Also, is there some other technique I can use to inspect all numpy generated objects? Thanks in advance. Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | phone: +45-29716388 | | DK-2500 Valby| email: mads.ip...@gmail.com | | Denmark | map : www.tinyurl.com/ns52fpa | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Tracking and inspecting numpy objects
Thanks to everybody for taking time to answer! Best regards, Mads On 15/09/14 12:11, Sebastian Berg wrote: On Mo, 2014-09-15 at 12:05 +0200, Eelco Hoogendoorn wrote: On Mon, Sep 15, 2014 at 11:55 AM, Sebastian Berg sebast...@sipsolutions.net wrote: On Mo, 2014-09-15 at 10:16 +0200, Mads Ipsen wrote: Hi, I am trying to inspect the reference count of numpy arrays generated by my application. Initially, I thought I could inspect the tracked objects using gc.get_objects(), but, with respect to numpy objects, the returned container is empty. For example: import numpy import gc data = numpy.ones(1024).reshape((32,32)) objs = [o for o in gc.get_objects() if isinstance(o, numpy.ndarray)] print objs# Prints empty list print gc.is_tracked(data) # Print False Why is this? Also, is there some other technique I can use to inspect all numpy generated objects? Two reasons. First of all, unless your array is an object arrays (or a structured one with objects in it), there are no objects to track. The array is a single python object without any referenced objects (except possibly its `arr.base`). Second of all -- and this is an issue -- numpy doesn't actually implement the traverse slot, so it won't even work for object arrays (numpy object arrays do not support circular garbage collection at this time, please feel free to implement it ;)). - Sebastian Does this answer why the ndarray object itself isn't tracked though? I must say I find this puzzling; the only thing I can think of is that the python compiler notices that data isn't used anymore after its creation, and deletes it right after its creation as an optimization, but that conflicts with my own experience of the GC. Not sure if it does, but my quick try and error says: In [15]: class T(tuple): : pass : In [16]: t = T() In [17]: objs = [o for o in gc.get_objects() if isinstance(o, T)] In [18]: objs Out[18]: [()] In [19]: a = 123. In [20]: objs = [o for o in gc.get_objects() if isinstance(o, float)] In [21]: objs Out[21]: [] So I guess nothing is tracked, unless it contains things, and numpy arrays don't say they can contain things (i.e. no traverse). - Sebastian ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | phone: +45-29716388 | | DK-2500 Valby| email: mads.ip...@gmail.com | | Denmark | map : www.tinyurl.com/ns52fpa | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Accessing irregular sized array data from C
Hi,
If you setup an M x N array like this
a = 1.0*numpy.arange(24).reshape(8,3)
you can access the data from a C function like this
void foo(PyObject * numpy_data)
{
// Get dimension and data pointer
int const m = static_castint(PyArray_DIMS(numpy_data)[0]);
int const n = static_castint(PyArray_DIMS(numpy_data)[1]);
double * const data = (double *) PyArray_DATA(numpy_data);
// Access data
...
}
Now, suppose I have an irregular shaped numpy array like this
a1 = numpy.array([ 1.0, 2.0, 3.0])
a2 = numpy.array([-2.0, 4.0])
a3 = numpy.array([5.0])
b = numpy.array([a1,a2,a3])
How can open up the doors to the array data of b on the C-side?
Best regards,
Mads
--
+-+
| Mads Ipsen |
+--+--+
| Gåsebæksvej 7, 4. tv | phone: +45-29716388 |
| DK-2500 Valby| email: mads.ip...@gmail.com |
| Denmark | map : www.tinyurl.com/ns52fpa |
+--+--+
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Accessing irregular sized array data from C
On 02/07/14 12:46, Julian Taylor wrote:
On Wed, Jul 2, 2014 at 12:15 PM, Mads Ipsen mads.ip...@gmail.com wrote:
Hi,
If you setup an M x N array like this
a = 1.0*numpy.arange(24).reshape(8,3)
you can access the data from a C function like this
void foo(PyObject * numpy_data)
{
// Get dimension and data pointer
int const m = static_castint(PyArray_DIMS(numpy_data)[0]);
int const n = static_castint(PyArray_DIMS(numpy_data)[1]);
double * const data = (double *) PyArray_DATA(numpy_data);
// Access data
...
}
Now, suppose I have an irregular shaped numpy array like this
a1 = numpy.array([ 1.0, 2.0, 3.0])
a2 = numpy.array([-2.0, 4.0])
a3 = numpy.array([5.0])
b = numpy.array([a1,a2,a3])
How can open up the doors to the array data of b on the C-side?
numpy does not directly support irregular shaped arrays (or ragged arrays).
If you look at the result of your example you will see this:
In [5]: b
Out[5]: array([array([ 1., 2., 3.]), array([-2., 4.]), array([
5.])], dtype=object)
b has datatype object, this means it is a 1d array containing more
array objects. Numpy does not directly know about the shapes or types
the sub arrays. It is not necessarily homogeneous anymore, but
compared to a regular python list you still have elementwise
operations (if the contained python objects support them) and it can
have multiple dimensions.
In C you would access such an array it like this:
PyArrayObject * const data = (PyArrayObject *) PyArray_DATA(numpy_data);
for (i=0; i PyArray_DIMS(numpy_data)[0]; i++) {
assert(PyArray_Check(data[i]));
double * const sub_data = (double *) PyArray_DATA(data[i]);
}
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Thanks - that'll get me going!
Best,
Mads
--
+-+
| Mads Ipsen |
+--+--+
| Gåsebæksvej 7, 4. tv | phone: +45-29716388 |
| DK-2500 Valby| email: mads.ip...@gmail.com |
| Denmark | map : www.tinyurl.com/ns52fpa |
+--+--+
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Fast decrementation of indices
Hi, Thanks to everybody for all you valuable responses. This approach by Rick White seems to nail it all down: b = np.array([ [0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 7, 8, 9, 10, 11], [5, 6, 1, 0, 2, 7, 3, 8, 1, 4, 9, 2, 10, 5, 3, 4, 11, 0, 0, 1, 2, 3, 4, 5] ]) a = [1,2,3,7,8] keepdata = np.ones(12, dtype=np.bool) keepdata[a] = False w = np.where(keepdata[b[0]] keepdata[b[1]]) newindex = keepdata.cumsum()-1 c = newindex[b[:,w[0]]] Also, I'd like to mention that I did think about using the graph module from SciPy. But the index bookkeeping done by numpy is in fact index pointers to memory location in a C++ driver - and not just labels. An when atoms are deleted, there memory chunks are also cleared, and therefore all pointers to these must be decremented. So using numpy for the bookkeeping seems a natural choice. Best regards, Mads On 02/03/2014 02:36 PM, Rick White wrote: b = np.array([ [0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 7, 8, 9, 10, 11], [5, 6, 1, 0, 2, 7, 3, 8, 1, 4, 9, 2, 10, 5, 3, 4, 11, 0, 0, 1, 2, 3, 4, 5] ]) a = [1,2,3,7,8] keepdata = np.ones(12, dtype=np.bool) keepdata[a] = False w = np.where(keepdata[b[0]] keepdata[b[1]]) newindex = keepdata.cumsum()-1 c = newindex[b[:,w[0]]] -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | phone: +45-29716388 | | DK-2500 Valby| email: mads.ip...@gmail.com | | Denmark | map : www.tinyurl.com/ns52fpa | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Fast decrementation of indices
Hi, I have run into a potential 'for loop' bottleneck. Let me outline: The following array describes bonds (connections) in a benzene molecule b = [[0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 7, 8, 9, 10, 11], [5, 6, 1, 0, 2, 7, 3, 8, 1, 4, 9, 2, 10, 5, 3, 4, 11, 0, 0, 1, 2, 3, 4, 5]] ie. bond 0 connects atoms 0 and 5, bond 1 connects atom 0 and 6, etc. In practical examples, the list can be much larger (N 100.000 connections. Suppose atoms with indices a = [1,2,3,7,8] are deleted, then all bonds connecting those atoms must be deleted. I achieve this doing i_0 = numpy.in1d(b[0], a) i_1 = numpy.in1d(b[1], a) b_i = numpy.where(i_0 | i_1)[0] b = b[:,~(i_0 | i_1)] If you find this approach lacking, feel free to comment. This results in the following updated bond list b = [[0, 0, 4, 4, 5, 5, 5, 6, 10, 11] [5, 6, 10, 5, 4, 11, 0, 0, 4, 5]] This list is however not correct: Since atoms [1,2,3,7,8] have been deleted, the remaining atoms with indices larger than the deleted atoms must be decremented. I do this as follows: for i in a: b = numpy.where(b i, bonds-1, bonds) (*) yielding the correct result b = [[0, 0, 1, 1, 2, 2, 2, 3, 5, 6], [2, 3, 5, 2, 1, 6, 0, 0, 1, 2]] The Python for loop in (*) may easily contain 50.000 iteration. Is there a smart way to utilize numpy functionality to avoid this? Thanks and best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | phone: +45-29716388 | | DK-2500 Valby| email: mads.ip...@gmail.com | | Denmark | map : www.tinyurl.com/ns52fpa | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Porting to the new C-API (1.7)
Hi, I've been using the numpy-1.6 C-API as part of a large C++ based OpenGL application. The C++ classes are exposed in Python by using SWIG, and utilize numpy arrays both as inputs to methods and method return values to the Python caller. To enable numpy in the SWIG generated Python module, the SWIG generated C++ file define #define PY_ARRAY_UNIQUE_SYMBOL PyArray_API whereas all other C++ files that need access to the numpy C-API contain #define NO_IMPORT_ARRAY #include numpy/arrayobject.h I have now updated to numpy-1.7, and receive warnings of the form #warning Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION This is basically fine with me, and I don't mind doing an update of my code to the new C-API. I have a few questions though: 1) Since I am apparently using the old API, where can I find a list of the deprecated things I use? That would make the upgrade easier. 2) Do I still have to use the PY_ARRAY_UNIQUE_SYMBOL approach when using the new C-API. 3) According to some websites you can do something like #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX This puzzles me a bit. Is there a doc somewhere where this whole thing is explained in detail. I must admit, its somewhat hard to grasp what's going on. Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] argsort
Hi, Thanks everybody for all the answers that make perfect sense when axis=0. Now suppose I want to sort the array in such a way that each row is sorted individually. Then I suppose I should do this: from numpy import * v = array([[4,3], [1,12], [23,7], [11,6], [8,9]]) idx = argsort(v, axis=1) idx is then [[1 0] [0 1] [1 0] [1 0] [0 1]] which makes sense, since these are the indices in an order that would sort each row. But when I try a[idx, variuos_additional_arguments] I just get strange results. Anybody that can point me towards the correct solution. Best regards, Mads On 01/15/2013 09:53 PM, eat wrote: Hi, On Tue, Jan 15, 2013 at 1:50 PM, Mads Ipsen madsip...@gmail.com mailto:madsip...@gmail.com wrote: Hi, I simply can't understand this. I'm trying to use argsort to produce indices that can be used to sort an array: from numpy import * indices = array([[4,3],[1,12],[23,7],[11,6],[8,9]]) args = argsort(indices, axis=0) print indices[args] gives: [[[ 1 12] [ 4 3]] [[ 4 3] [11 6]] [[ 8 9] [23 7]] [[11 6] [ 8 9]] [[23 7] [ 1 12]]] I thought this should produce a sorted version of the indices array. Any help is appreciated. Perhaps these three different point of views will help you a little bit more to move on: In []: x Out[]: array([[ 4, 3], [ 1, 12], [23, 7], [11, 6], [ 8, 9]]) In []: ind= x.argsort(axis= 0) In []: ind Out[]: array([[1, 0], [0, 3], [4, 2], [3, 4], [2, 1]]) In []: x[ind[:, 0]] Out[]: array([[ 1, 12], [ 4, 3], [ 8, 9], [11, 6], [23, 7]]) In []: x[ind[:, 1]] Out[]: array([[ 4, 3], [11, 6], [23, 7], [ 8, 9], [ 1, 12]]) In []: x[ind, [0, 1]] Out[]: array([[ 1, 3], [ 4, 6], [ 8, 7], [11, 9], [23, 12]]) -eat Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone:+45-29716388 tel:%2B45-29716388 | | Denmark | email:mads.ip...@gmail.com mailto:mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org mailto:NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] argsort
Hi, I simply can't understand this. I'm trying to use argsort to produce indices that can be used to sort an array: from numpy import * indices = array([[4,3],[1,12],[23,7],[11,6],[8,9]]) args = argsort(indices, axis=0) print indices[args] gives: [[[ 1 12] [ 4 3]] [[ 4 3] [11 6]] [[ 8 9] [23 7]] [[11 6] [ 8 9]] [[23 7] [ 1 12]]] I thought this should produce a sorted version of the indices array. Any help is appreciated. Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] int and long issues
Hi, I find this to be a little strange: x = numpy.arange(10) isinstance(x[0],int) gives True y = numpy.where(x 5)[0] isinstance(y[0],int) gives False isinstance(y[0],long) gives True Specs: Python 2.7.2, numpy-1.6.1, Win7, 64 bit Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] int and long issues
Sebastian - thanks - very helpful. Best regards, Mads On 10/01/2013 12:06, Sebastian Berg wrote: On Thu, 2013-01-10 at 11:32 +0100, Mads Ipsen wrote: Hi, I find this to be a little strange: x = numpy.arange(10) isinstance(x[0],int) gives True y = numpy.where(x 5)[0] isinstance(y[0],int) gives False isinstance(y[0],long) Check what type(x[0])/type(y[0]) prints, I expect these are very different, because the default integer type and the integer type used for indexing (addressing memory in general) are not necessarily the same. And because of that, `y[0]` probably simply isn't compatible to the datatype of a python integer for your hardware and OS (for example for me, your code works). So on python 2 (python 3 abolishes int and makes long the only integer, so this should work as expected there) you have to just check both even in the python context, because you can never really know (there may be some nice trick for that, but not sure). And if you want to allow for rare 0d arrays as well (well they are very rare admittingly)... it gets even a bit hairier. gives True Specs: Python 2.7.2, numpy-1.6.1, Win7, 64 bit Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Slices from an index list
Hi, Suppose a have an array of indices, say indices = [0,1,2,3,5,7,8,9,10,12,13,14] Then the following slices a = slice(0,4) b = slice(4,5) c = slice(5,9) d = slice(9,12) provide information about all the consecutive parts of the index list. Given the list of indices, is there some nifty numpy function that can generate the above slices for me (or their start and stop values)? Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Slices from an index list
On 11/04/2012 13:42, Warren Weckesser wrote: On Wed, Apr 11, 2012 at 4:28 AM, Mads Ipsen madsip...@gmail.com mailto:madsip...@gmail.com wrote: Hi, Suppose a have an array of indices, say indices = [0,1,2,3,5,7,8,9,10,12,13,14] Then the following slices a = slice(0,4) b = slice(4,5) c = slice(5,9) d = slice(9,12) provide information about all the consecutive parts of the index list. Given the list of indices, is there some nifty numpy function that can generate the above slices for me (or their start and stop values)? Here's one way you could do it: In [43]: indices = [0,1,2,3,5,7,8,9,10,12,13,14] In [44]: jumps = where(diff(indices) != 1)[0] + 1 In [45]: starts = hstack((0, jumps)) In [46]: ends = hstack((jumps, len(indices))) In [47]: slices = [slice(start, end) for start, end in zip(starts, ends)] In [48]: slices Out[48]: [slice(0, 4, None), slice(4, 5, None), slice(5, 9, None), slice(9, 12, None)] Warren ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Thanks - very helpful! -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Slices from an index list
On 11/04/2012 14:19, Zachary Pincus wrote: Here's one way you could do it: In [43]: indices = [0,1,2,3,5,7,8,9,10,12,13,14] In [44]: jumps = where(diff(indices) != 1)[0] + 1 In [45]: starts = hstack((0, jumps)) In [46]: ends = hstack((jumps, len(indices))) In [47]: slices = [slice(start, end) for start, end in zip(starts, ends)] In [48]: slices Out[48]: [slice(0, 4, None), slice(4, 5, None), slice(5, 9, None), slice(9, 12, None)] If you're only going to use the slices to divide up the list, you could use numpy.split and skip creating the slice objects: indices = [0,1,2,3,5,7,8,9,10,12,13,14] jumps = numpy.where(numpy.diff(indices) != 1)[0] + 1 numpy.split(indices, jumps) giving: [array([0, 1, 2, 3]), array([5]), array([ 7, 8, 9, 10]), array([12, 13, 14])] Zach (btw, Warren, the method to calculate the jumps is cute. I'll have to remember that.) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Thanks - very helpful! -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Minus zero
Here's an array a = [[ -2.66453526e-15, 4.49564793e-02, 1.14401980e+00], [ 2.02475000e+00, 2.06970648e+00, 1.14401980e+00], [ 2.02475000e+00, 4.49564793e-02, 3.16876980e+00], [ -2.66453526e-15, 2.06970648e+00, 3.16876980e+00], [ -2.66453526e-15, 4.49564793e-02, 5.19351980e+00], [ 2.02475000e+00, 2.06970648e+00, 5.19351980e+00], [ 2.02475000e+00, 4.49564793e-02, 7.21826980e+00], [ 0.e+00, 2.02475000e+00, 7.08662500e+00]] Now, call array_repr(a, precision=12, suppress_small=True) which gives the output array([[-0. , 0.0449564793, 1.1440198 ], [ 2.02475 , 2.06970648 , 1.1440198 ], [ 2.02475 , 0.0449564793, 3.1687698 ], [-0. , 2.06970648 , 3.1687698 ], [-0. , 0.0449564793, 5.1935198 ], [ 2.02475 , 2.06970648 , 5.1935198 ], [ 2.02475 , 0.0449564793, 7.2182698 ], [ 0. , 2.02475 , 7.086625]]) * Since the flag 'suppress_small=True' is used, I'd expect a '0.' instead of a '-0.' or maybe this is intentional. * Also, wouldn't '0.0' instead of '0.' be more readable (and there's certainly space enough to add it)? Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] _import_array()
Hi,
I have C++ module (OpenGL) that extracts data from numpy arrays. The
interface is pure read-only: It never returns any Python objects but
only extracts data from numpy arrays. Eg:
#include numpy/arrayobject.h
void PrimitiveManager::deleteAtoms(PyObject * numpy_indices)
{
// Extract number of indices
int const n = static_castint(PyArray_DIMS(numpy_indices)[0]);
long * const indices = (long *) PyArray_DATA(numpy_indices);
// Delete atoms in buffer
for (int i = 0; i n; ++i)
{
// Do stuff
}
}
Now, when I compile the code with g++, I get the following warning:
numpy/core/include/numpy/__multiarray_api.h:1532: warning: 'int
_import_array()' defined but not used
Do I need to call '_import_array()' somewhere? Am I doing something
potentially nasty?
Best regards,
Mads
--
+-+
| Mads Ipsen |
+--+--+
| Gåsebæksvej 7, 4. tv | |
| DK-2500 Valby| phone: +45-29716388 |
| Denmark | email: mads.ip...@gmail.com |
+--+--+
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] _import_array()
Hi,
The C++ module here is a class that's used by an OpenGL window, to
extract data from numpy arrays and basically draw molecules whose
coordinates are stored in numpy arrays.
The C++ module is accessed from Python using wrappers generated by swig.
Our application may contain many active OpenGL windows, where each one
of them contains an instance of the swig wrapped C++ module. So the
question is then:
* Should the constructor of each instance of the C++ class call
import_array()?
* Should import_array() be called every time a method in the C++ class
handles a numpy structure?
* Should import_array() only be called one time, namely when the main
application is started?
Best regards,
Mads
On 14/02/2012 10:09, Travis Oliphant wrote:
Technically, when you write an extension module you really should use
import_array(); in the init method of the extensions module. This
ensures that the C-API is loaded so that the API -table is available
if your C++ code uses the C-API at all.
In this case you are just using some #defines that access the NumPy
array structure, so it works without the import_array(). However,
this could change in future releases (i.e. PyArray_DIMS and
PyArray_DATA could become functions that are looked up in an API-table
that must be loaded by import_array() ).
Best regards,
-Travis
On Feb 14, 2012, at 3:03 AM, Mads Ipsen wrote:
Hi,
I have C++ module (OpenGL) that extracts data from numpy arrays. The
interface is pure read-only: It never returns any Python objects but
only extracts data from numpy arrays. Eg:
#include numpy/arrayobject.h
void PrimitiveManager::deleteAtoms(PyObject * numpy_indices)
{
// Extract number of indices
int const n = static_castint(PyArray_DIMS(numpy_indices)[0]);
long * const indices = (long *) PyArray_DATA(numpy_indices);
// Delete atoms in buffer
for (int i = 0; i n; ++i)
{
// Do stuff
}
}
Now, when I compile the code with g++, I get the following warning:
numpy/core/include/numpy/__multiarray_api.h:1532: warning: ‘int
_import_array()’ defined but not used
Do I need to call '_import_array()' somewhere? Am I doing something
potentially nasty?
Best regards,
Mads
--
+-+
| Mads Ipsen |
+--+--+
| Gåsebæksvej 7, 4. tv | |
| DK-2500 Valby| phone: +45-29716388 |
| Denmark | email:mads.ip...@gmail.com |
+--+--+
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org mailto:NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
--
+-+
| Mads Ipsen |
+--+--+
| Gåsebæksvej 7, 4. tv | |
| DK-2500 Valby| phone: +45-29716388 |
| Denmark | email: mads.ip...@gmail.com |
+--+--+
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] _import_array()
On 14/02/2012 10:30, Pauli Virtanen wrote:
14.02.2012 10:20, Mads Ipsen kirjoitti:
[clip]
* Should import_array() only be called one time, namely when the main
application is started?
It should be called once when the application is started, before you do
any other Numpy-using operations.
http://docs.scipy.org/doc/numpy/reference/c-api.array.html#import_array
This is what we have in our swig.i file:
%init
%{
import_array();
%}
so I guess we are doing it the right way.
But I still get the warning, when the code is compiled.
Mads
--
+-+
| Mads Ipsen |
+--+--+
| Gåsebæksvej 7, 4. tv | |
| DK-2500 Valby| phone: +45-29716388 |
| Denmark | email: mads.ip...@gmail.com |
+--+--+
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Unexpected reorganization of internal data
On 31/01/2012 18:23, Chris Barker wrote: On Tue, Jan 31, 2012 at 6:14 AM, Malcolm Reynolds malcolm.reyno...@gmail.com wrote: Not exactly an answer to your question, but I can highly recommend using Boost.python, PyUblas and Ublas for your C++ vectors and matrices. It gives you a really good interface on the C++ side to numpy arrays and matrices, which can be passed in both directions over the language threshold with no copying. or use Cython... If I had to guess I'd say sometimes when transposing numpy simply sets a flag internally to avoid copying the data, but in some cases (such as perhaps when multiplication needs to take place) the data has to be placed in a new object. good guess: V = numpy.dot(R, U.transpose()).transpose() a array([[1, 2], [3, 4], [5, 6]]) a.flags C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False b = a.transpose() b.flags C_CONTIGUOUS : False F_CONTIGUOUS : True OWNDATA : False WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False so the transpose() simple re-arranges the strides to Fortran order, rather than changing anything in memory. np.dot() produces a new array, so it is C-contiguous, then you transpose it, so you get a fortran-ordered array. Now when I call my C++ function from the Python side, all the data in V is printed, but it has been transposed. as mentioned, if you are working with arrays in C++ (or fortran, orC, or...) and need to count on the ordering of the data, you need to check it in your extension code. There are utilities for this. However, if I do: V = numpy.array(U.transpose()).transpose() right: In [7]: a.flags Out[7]: C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False In [8]: a.transpose().flags Out[8]: C_CONTIGUOUS : False F_CONTIGUOUS : True OWNDATA : False WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False In [9]: np.array( a.transpose() ).flags Out[9]: C_CONTIGUOUS : False F_CONTIGUOUS : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False so the np.array call doesn't re-arrange the order if it doesn't need to. If you want to force it, you can specify the order: In [10]: np.array( a.transpose(), order='C' ).flags Out[10]: C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False (note: this does surprise me a bit, as it is making a copy, but there you go -- if order matters, specify it) In general, numpy does a lot of things for the sake of efficiency -- avoiding copies when it can, for instance -- this give efficiency and flexibility, but you do need to be careful, particularly when interfacing with the binary data directly. -Chris Thanks for all the answers to my question. Helped a lot. Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Unexpected reorganization of internal data
Hi,
I am confused. Here's the reason:
The following structure is a representation of N points in 3D space:
U = numpy.array([[x1,y1,z1], [x1,y1,z1],...,[xn,yn,zn]])
So the array U has shape (N,3). This order makes sense to me since U[i]
will give you the i'th point in the set. Now, I want to pass this array
to a C++ function that does some stuff with the points. Here's how I do
that
void Foo::doStuff(int n, PyObject * numpy_data)
{
// Get pointer to data
double * const positions = (double *) PyArray_DATA(numpy_data);
// Print positions
for (int i=0; in; ++i)
{
float x = static_castfloat(positions[3*i+0])
float y = static_castfloat(positions[3*i+1])
float z = static_castfloat(positions[3*i+2])
printf(Pos[%d] = %f %f %f\n, x, y, z);
}
}
When I call this routine, using a swig wrapped Python interface to the
C++ class, everything prints out nice.
Now, I want to apply a rotation to all the positions. So I set up some
rotation matrix R like this:
R = numpy.array([[r11,r12,r13],
[r21,r22,r23],
[r31,r32,r33]])
To apply the matrix to the data in one crunch, I do
V = numpy.dot(R, U.transpose()).transpose()
Now when I call my C++ function from the Python side, all the data in V
is printed, but it has been transposed. So apparently the internal data
structure handled by numpy has been reorganized, even though I called
transpose() twice, which I would expect to cancel out each other.
However, if I do:
V = numpy.array(U.transpose()).transpose()
and call the C++ routine, everything is perfectly fine, ie. the data
structure is as expected.
What went wrong?
Best regards,
Mads
--
+-+
| Mads Ipsen |
+--+--+
| Gåsebæksvej 7, 4. tv | |
| DK-2500 Valby| phone: +45-29716388 |
| Denmark | email: mads.ip...@gmail.com |
+--+--+
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Index update
Hi, Suppose you have N items, say N = 10. Now a subset of these items are selected given by a list A of indices. Lets say that items A = [2,5,7] are selected. Assume now that you delete some of the items given by the indices S = [1,4,8]. This means that the list of indices A must be updated, since items have been deleted. For this particular case the updated selection list A becomes A = [1,3,5]. Is there some smart numpy way of doing this index update of the selected items in A without looping? Typically N is large. Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Warning related to __multiarray_api.h
Hi, I don't know if this is of importance, but when I compile code using the numpy C API, I get the warning: site-packages/numpy/core/include/numpy/__multiarray_api.h:1532: warning: 'int _import_array()' defined but not used Might be worth cleaning it up. Best regards, Mads ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] dtype
Hi, Where can I find a complete and exhaustive description of the dtype syntax and arguments. For example, something like a = arange(4, dtype='=H') seems hard to extract from the documentation that I can find on the web and the numpy docstrings. Best regards Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Build warnings
Hi, I am currently building numpy 1.6.1 on Ubuntu 11.04 using gcc 4.4.5. Some compiler warnings are printed during the compilation stage. Is it of any interest to post these somewhere in order for some of the core developers to fix these? Best regards, Mads -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Failure to build numpy 1.6.1
Hi, I am trying to build numpy-1.6.1 with the following gcc compiler specs: Reading specs from /usr/lib/gcc/x86_64-redhat-linux/3.4.6/specs Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-java-awt=gtk --host=x86_64-redhat-linux Thread model: posix gcc version 3.4.6 20060404 (Red Hat 3.4.6-11) I get the following error (any clues at what goes wrong)? creating build/temp.linux-x86_64-2.7/numpy/core/src/multiarray compile options: '-Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/include -I/home/quantum/quantumnotes/qw-control/quantumsource/external-libs/build/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/multiarray -Ibuild/src.linux-x86_64-2.7/numpy/core/src/umath -c' gcc: numpy/core/src/multiarray/multiarraymodule_onefile.c numpy/core/src/multiarray/descriptor.c: In function `_convert_divisor_to_multiple': numpy/core/src/multiarray/descriptor.c:606: warning: 'q' might be used uninitialized in this function numpy/core/src/multiarray/einsum.c.src: In function `float_sum_of_products_contig_outstride0_one': numpy/core/src/multiarray/einsum.c.src:852: error: unrecognizable insn: (insn:HI 440 228 481 14 /usr/lib/gcc/x86_64-redhat-linux/3.4.6/include/xmmintrin.h:915 (set (reg:SF 148) (vec_select:SF (reg/v:V4SF 67 [ accum_sse ]) (parallel [ (const_int 0 [0x0]) ]))) -1 (insn_list 213 (nil)) (nil)) numpy/core/src/multiarray/einsum.c.src:852: internal compiler error: in extract_insn, at recog.c:2083 Please submit a full bug report, with preprocessed source if appropriate. See URL:http://bugzilla.redhat.com/bugzilla for instructions. Preprocessed source stored into /tmp/ccXaPpf8.out file, please attach this to your bugreport. numpy/core/src/multiarray/descriptor.c: In function `_convert_divisor_to_multiple': numpy/core/src/multiarray/descriptor.c:606: warning: 'q' might be used uninitialized in this function numpy/core/src/multiarray/einsum.c.src: In function `float_sum_of_products_contig_outstride0_one': numpy/core/src/multiarray/einsum.c.src:852: error: unrecognizable insn: (insn:HI 440 228 481 14 /usr/lib/gcc/x86_64-redhat-linux/3.4.6/include/xmmintrin.h:915 (set (reg:SF 148) (vec_select:SF (reg/v:V4SF 67 [ accum_sse ]) (parallel [ (const_int 0 [0x0]) ]))) -1 (insn_list 213 (nil)) (nil)) numpy/core/src/multiarray/einsum.c.src:852: internal compiler error: in extract_insn, at recog.c:2083 Please submit a full bug report, with preprocessed source if appropriate. See URL:http://bugzilla.redhat.com/bugzilla for instructions. Preprocessed source stored into /tmp/ccXaPpf8.out file, please attach this to your bugreport. error: Command gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/include -I/home/quantum/quantumnotes/qw-control/quantumsource/external-libs/build/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/multiarray -Ibuild/src.linux-x86_64-2.7/numpy/core/src/umath -c numpy/core/src/multiarray/multiarraymodule_onefile.c -o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/multiarraymodule_onefile.o failed with exit status 1 make: *** [/home/quantum/quantumnotes/qw-control/quantumsource/external-libs/src/numpy-1.6.1/make-stamp] Error 1 -- +-+ | Mads Ipsen | +--+--+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby| phone: +45-29716388 | | Denmark | email: mads.ip...@gmail.com | +--+--+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Tuple outer product?
Robert Kern wrote: On Fri, Sep 25, 2009 at 17:38, Mads Ipsen m...@comxnet.dk wrote: Yes, but it should also work for [2.1,3.2,4.5] combined with [4.6,-2.3,5.6] - forgot to tell that. In [5]: np.transpose(np.meshgrid([2.1,3.2,4.5], [4.6,-2.3,5.6])) Out[5]: array([[[ 2.1, 4.6], [ 2.1, -2.3], [ 2.1, 5.6]], [[ 3.2, 4.6], [ 3.2, -2.3], [ 3.2, 5.6]], [[ 4.5, 4.6], [ 4.5, -2.3], [ 4.5, 5.6]]]) Point taken :-) -- ++ | Mads Ipsen, Scientific developer | +--+-+ | QuantumWise A/S | phone: +45-29716388 | | Nørresøgade 27A | www:www.quantumwise.com | | DK-1370 Copenhagen, Denmark | email: m...@quantumwise.com | +--+-+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Tuple outer product?
Is there a numpy operation on two arrays, say [1,2,3] and [4,5,6], that will yield: [[(1,4),(1,5),(1,6)],[(2,4),(2,5),(2,6)],[(3,4),(3,5),(3,6)]] Any suggestions are most welcome. Mads -- ++ | Mads Ipsen, Scientific developer | +--+-+ | QuantumWise A/S | phone: +45-29716388 | | Nørresøgade 27A | www:www.quantumwise.com | | DK-1370 Copenhagen, Denmark | email: m...@quantumwise.com | +--+-+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Tuple outer product?
Gökhan Sever wrote: On Fri, Sep 25, 2009 at 12:45 PM, Mads Ipsen m...@comxnet.dk mailto:m...@comxnet.dk wrote: Is there a numpy operation on two arrays, say [1,2,3] and [4,5,6], that will yield: [[(1,4),(1,5),(1,6)],[(2,4),(2,5),(2,6)],[(3,4),(3,5),(3,6)]] Any suggestions are most welcome. Mads I don't know if there is a function in numpy library, but it is a simple one isn't it? ipython --pylab I[1]: a = array([1,2,3]) I[2]: b = array([4,5,6]) I[3]: [zip(ones(a.size, dtype=int)*a[i], b) for i in range(len(a))] O[3]: [[(1, 4), (1, 5), (1, 6)], [(2, 4), (2, 5), (2, 6)], [(3, 4), (3, 5), (3, 6)]] -- ++ | Mads Ipsen, Scientific developer | +--+-+ | QuantumWise A/S | phone: +45-29716388 | | Nørresøgade 27A | www:www.quantumwise.com http://www.quantumwise.com | | DK-1370 Copenhagen, Denmark | email: m...@quantumwise.com mailto:m...@quantumwise.com | +--+-+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org mailto:NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Gökhan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Sure, and you could do the same thing using two nested for loops. But its bound to be slow when the arrays become large (and they will). The question is whether it can be done in pure numpy. An output like [[[1,4],[1,5],[1,6]],[[2,4],[2,5],[2,6]],[[3,4],[3,5],[3,6]]] is also OK. Mads -- ++ | Mads Ipsen, Scientific developer | +--+-+ | QuantumWise A/S | phone: +45-29716388 | | Nørresøgade 27A | www:www.quantumwise.com | | DK-1370 Copenhagen, Denmark | email: m...@quantumwise.com | +--+-+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Tuple outer product?
Robert Kern wrote: On Fri, Sep 25, 2009 at 14:19, Alan G Isaac ais...@american.edu wrote: I do not see what is wrong with itertools.product, but if you hate it, you can use numpy.meshgrid: np.array(np.meshgrid([1,2,3],[4,5,6])).transpose() array([[[1, 4], [1, 5], [1, 6]], [[2, 4], [2, 5], [2, 6]], [[3, 4], [3, 5], [3, 6]]]) If you need more than two item sets, or are using Python 2.5: import numpy as np def cartesian_product(*items): items = map(np.asarray, items) lengths = map(len, items) n = np.arange(np.product(lengths)) results = [] for i in range(-1, -len(items)-1, -1): j = n % lengths[i] results.insert(0, items[i][j]) n -= j n //= lengths[i] results = np.column_stack(results) results.shape = tuple(lengths + [len(items)]) return results The final shape manipulations are, of course, optional. Thanks for all the suggestions. Came up with this, which I think I'll stick with a = numpy.array([1,2,3]) b = numpy.array([4,5,6]) (n,m) = (a.shape[0],b.shape[0]) a = numpy.repeat(a,m).reshape(n,m) b = numpy.repeat(b,n).reshape(m,n).transpose() ab = numpy.dstack((a,b)) print ab.tolist() [[[1, 4], [1, 5], [1, 6]], [[2, 4], [2, 5], [2, 6]], [[3, 4], [3, 5], [3, 6]]] -- ++ | Mads Ipsen, Scientific developer | +--+-+ | QuantumWise A/S | phone: +45-29716388 | | Nørresøgade 27A | www:www.quantumwise.com | | DK-1370 Copenhagen, Denmark | email: m...@quantumwise.com | +--+-+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Tuple outer product?
Alan G Isaac wrote: On 9/25/2009 4:01 PM, Mads Ipsen wrote: a = numpy.array([1,2,3]) b = numpy.array([4,5,6]) (n,m) = (a.shape[0],b.shape[0]) a = numpy.repeat(a,m).reshape(n,m) b = numpy.repeat(b,n).reshape(m,n).transpose() ab = numpy.dstack((a,b)) print ab.tolist() That's just a slow implementation of meshgrid: np.meshgrid(a,b).transpose().tolist() Gives you the same thing. Alan Isaac ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Yes, but it should also work for [2.1,3.2,4.5] combined with [4.6,-2.3,5.6] - forgot to tell that. Mads -- ++ | Mads Ipsen, Scientific developer | +--+-+ | QuantumWise A/S | phone: +45-29716388 | | Nørresøgade 27A | www:www.quantumwise.com | | DK-1370 Copenhagen, Denmark | email: m...@quantumwise.com | +--+-+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Error in header file - wrong mailing list?
Pauli Virtanen wrote: su, 2009-09-13 kello 01:47 +0300, Pauli Virtanen kirjoitti: [clip] The error you get from the comma at the end of the enum must be because you have -pedantic -Werror in your CFLAGS. Just unset CFLAGS or so before compilation. Yes, comma at the end of enum list is not strictly valid C89, although it is valid C99 and already passes for most compilers. Another possibility is that these flags come from /usr/lib/pythonX.Y/config/Makefile -- in that case it's maybe possible to override one of the the BASECFLAGS etc. variables with environment vars. Also, I see that OPT=-std=c89 -pedantic -Werror python setup.py won't succeed in current SVN, because apparently the configuration detection code is not strict C89. Well, I don't know you consider this as a valid argument, but to me its a matter of removing a single comma, which will make the source less sensitive to compilers and compiler flags. Mads -- ++ | Mads Ipsen, Scientific developer | +--+-+ | QuantumWise A/S | phone: +45-29716388 | | Nørresøgade 27A | www:www.quantumwise.com | | DK-1370 Copenhagen, Denmark | email: m...@quantumwise.com | +--+-+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Error in header file - wrong mailing list?
Hey, I recently posted a bug related to a compile error in the header file 'npy_common.h' but have received no responses so far. Am I posting this in the wrong mailing list? Best regards, Mads -- ++ | Mads Ipsen, Scientific developer | +--+-+ | QuantumWise A/S | phone: +45-29716388 | | Nørresøgade 27A | www:www.quantumwise.com | | DK-1370 Copenhagen, Denmark | email: m...@quantumwise.com | +--+-+ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] error: comma at end of enumerator list
Hey,
When I try to compile a swig based interface to NumPy, I get the error:
lib/python2.6/site-packages/numpy/core/include/numpy/npy_common.h:11:
error: comma at end of enumerator list
In npy_common.h, changing
/* enums for detected endianness */
enum {
NPY_CPU_UNKNOWN_ENDIAN,
NPY_CPU_LITTLE,
NPY_CPU_BIG,
};
to
/* enums for detected endianness */
enum {
NPY_CPU_UNKNOWN_ENDIAN,
NPY_CPU_LITTLE,
NPY_CPU_BIG
};
fixes the issue. I believe this should be fixed. At least we cannot
built our software without the above fix.
System info:
gcc version 4.3.2 (Ubuntu 4.3.2-1ubuntu12)
Ubuntu 8.10
Best regards,
Mads
--
++
| Mads Ipsen, Scientific developer |
+--+-+
| QuantumWise A/S | phone: +45-29716388 |
| Nørresøgade 27A | www:www.quantumwise.com |
| DK-1370 Copenhagen, Denmark | email: m...@quantumwise.com |
+--+-+
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
