Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
Ah! So much ado about nothing. What I was looking for was in fact: B[A_idx][:,A_idx] ... it's even explained in the the NumPy for Matlab Users doc on scipy.org /Thank you ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On 3/20/07, David Koch <[EMAIL PROTECTED]> wrote:
> And by the way - whenever I do a .__doc__ all newline characters
> are printed as "\n" in the python console ... how do I change that?
The easiest way to access the doc strings is to type help()
in the python interpreter, or ? in ipython
().
The reason you are seeing the escaped newlines is that by directly
accessing the __doc__ attribute, you are getting the string which the
interpreter is kind enough to display in repr form, but you are not
explicitly printing it (in str form). Compare:
>>> print repr('hello\nworld')
'hello\nworld'
>>> print str('hello\nworld')
hello
world
So that if you said print .__doc__, you would not see the
escaped newlines.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
I will consider it Sven, I thought it was a good idea to collect everything which had to do with Matlab -> Python in one thread. Anyway, Specifically, I was looking for an equivalent to Matlab's "sprand" which allows one to create sparse normally distributed matrices. The function also accepts a "density" parameter. And by the way - whenever I do a .__doc__ all newline characters are printed as "\n" in the python console ... how do I change that? /David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
David Koch schrieb: > Hi, > > naive question - how do I get an overview over everything to do with > "sparse functionality" in SciPy 0.5.2 ... I can't find any documentation > anywhere. > First of all I would recommend to start a new and properly named thread for that good luck, sven ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
Hi, naive question - how do I get an overview over everything to do with "sparse functionality" in SciPy 0.5.2 ... I can't find any documentation anywhere. /David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On Thu, Mar 15, 2007 at 12:26:10PM +0100, David Koch wrote: > Next thing, I have > > A_Idx = array([1, 0]) > > and B is a matrix. How would I select the four elements in B indicated by > "meshgrid(A_Idx)", that ist: B[1,1], B[1,0], B[0,1], B[0,0] > In Matlab you would simply use B(A_idx, A_idx) whereas in NumPy B[A_Idx, > A_idx] > returns B[1,1] and B[0,0] only. > > I read that A.take(A_idx).take(A_idx, axis = 1) should work but it > doesn't. In [12]: B[N.ix_(A_idx,A_idx)] Out[12]: array([[4, 3], [2, 1]]) Cheers Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On 3/15/07, David Koch <[EMAIL PROTECTED]> wrote: ... NumPy equiv for Matlab B(A_idx, A_Idx) Ok, I did like this: A_Idx = array([1, 0]) B = random.randn(3,3) rowInd = kron(ones((1, len(A_Idx)), int), A_Idx[:, newaxis]) colInd = kron(ones((len(A_Idx), 1), int), A_Idx) B[rowInd, colInd] - returns the equivalent of Matlabs B(A_Idx, A_Idx) ... that cannot possibly be the easiest way :-/ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On 3/14/07, Gael Varoquaux <[EMAIL PROTECTED]> wrote: I definitely second this comment. Using arrays when you are trying to append a lot of data is using the wrong data format. And the code is so much more readable with lists. Thank you, I will consider it, Next thing, I have A_Idx = array([1, 0]) and B is a matrix. How would I select the four elements in B indicated by "meshgrid(A_Idx)", that ist: B[1,1], B[1,0], B[0,1], B[0,0] In Matlab you would simply use B(A_idx, A_idx) whereas in NumPy B[A_Idx, A_idx] returns B[1,1] and B[0,0] only. I read that A.take(A_idx).take(A_idx, axis = 1) should work but it doesn't. Thank you, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On Wed, Mar 14, 2007 at 04:11:43PM +0100, Sturla Molden wrote: > On 3/14/2007 2:46 PM, Robert Cimrman wrote: > > a = [] > > while ... > > a.append( scalar ) > > a = array( a ) > While it may help, growing Python lists is also an O(N) process. > One can reduce the amount of allocations by preallocating an ndarray of > a certain size (e.g. 1024 scalars), filling it up, and storing it in a > linked list. Finally, the stored arrays are retrieved as a single > contiguous array. Example code below (cf. class scalar_list). > Sturla Molden I definitely second this comment. Using arrays when you are trying to append a lot of data is using the wrong data format. And the code is so much more readable with lists. If you want to do something even nicer you might want to use a generator, possibly one created with the "yield" keyword. -- Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On 3/14/07, Sturla Molden <[EMAIL PROTECTED]> wrote: On 3/14/2007 2:46 PM, Robert Cimrman wrote: > a = [] > while ... > a.append( scalar ) > a = array( a ) While it may help, growing Python lists is also an O(N) process. This may just be a terminology problem, but just to be clear, appending to a Python list is amortized (on average) constant time (O(1)), not O(N). One can reduce the amount of allocations by preallocating an ndarray of a certain size (e.g. 1024 scalars), filling it up, and storing it in a linked list. Finally, the stored arrays are retrieved as a single contiguous array. Example code below (cf. class scalar_list). Sturla Molden import numpy class ndarray_list: """ a single linked list of numpy ndarrays.""" class node: def __init__(self, data): self.next = None self.data = data def __init__(self): self.head = None self.tail = None self.len = 0 def append(self, data): tmp = self.node(data) if self.tail == None: self.tail = tmp self.head = tmp self.len = len(data) else: self.tail.next = tmp self.tail = tmp self.len += len(data) def length(self): return self.len def flatten(self, dtype=float): tmp = numpy.empty(self.len, dtype=dtype) cur = self.head idx0 = 0 while cur: tmp[idx0:idx0+len(cur.data)] = cur.data idx0 += len(cur.data) cur = cur.next return tmp class scalar_list: """ a single linked list of numpy scalars.""" def __init__(self, size=1024, dtype=float): self.cur = 0 self.size = size self.dtype = dtype self.arr = numpy.empty(size,dtype) self.arrlist = ndarray_list() def append(self, scalar): cur = self.cur self.arr[cur] = scalar self.cur += 1 if self.cur == self.size: self.arrlist.append(self.arr) self.arr = numpy.empty(self.size,self.dtype) self.cur = 0 def array(self): if self.cur: self.arrlist.append(self.arr[:self.cur]) retval = self.arrlist.flatten(self.dtype) self.cur = 0 self.arr = numpy.empty(self.size,self.dtype) self.arrlist = ndarray_list() self.arrlist.append(retval.copy()) return retval ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion -- //=][=\\ [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On 3/14/2007 2:46 PM, Robert Cimrman wrote: > a = [] > while ... > a.append( scalar ) > a = array( a ) While it may help, growing Python lists is also an O(N) process. One can reduce the amount of allocations by preallocating an ndarray of a certain size (e.g. 1024 scalars), filling it up, and storing it in a linked list. Finally, the stored arrays are retrieved as a single contiguous array. Example code below (cf. class scalar_list). Sturla Molden import numpy class ndarray_list: """ a single linked list of numpy ndarrays.""" class node: def __init__(self, data): self.next = None self.data = data def __init__(self): self.head = None self.tail = None self.len = 0 def append(self, data): tmp = self.node(data) if self.tail == None: self.tail = tmp self.head = tmp self.len = len(data) else: self.tail.next = tmp self.tail = tmp self.len += len(data) def length(self): return self.len def flatten(self, dtype=float): tmp = numpy.empty(self.len, dtype=dtype) cur = self.head idx0 = 0 while cur: tmp[idx0:idx0+len(cur.data)] = cur.data idx0 += len(cur.data) cur = cur.next return tmp class scalar_list: """ a single linked list of numpy scalars.""" def __init__(self, size=1024, dtype=float): self.cur = 0 self.size = size self.dtype = dtype self.arr = numpy.empty(size,dtype) self.arrlist = ndarray_list() def append(self, scalar): cur = self.cur self.arr[cur] = scalar self.cur += 1 if self.cur == self.size: self.arrlist.append(self.arr) self.arr = numpy.empty(self.size,self.dtype) self.cur = 0 def array(self): if self.cur: self.arrlist.append(self.arr[:self.cur]) retval = self.arrlist.flatten(self.dtype) self.cur = 0 self.arr = numpy.empty(self.size,self.dtype) self.arrlist = ndarray_list() self.arrlist.append(retval.copy()) return retval ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On 3/14/07, David Koch <[EMAIL PROTECTED]> wrote: On 3/14/07, Sven Schreiber <[EMAIL PROTECTED]> wrote: > > > > If you want a 1d-array in the end you could try empty(0) to start with, > and then do hstack((A, your_scalar)) or something like that. Depending on what your generating routine looks like, you might also try "a = fromiter(...)" where you package up your scalar source as an iterator. -- //=][=\\ [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On 3/14/07, Sven Schreiber <[EMAIL PROTECTED]> wrote: If you want a 1d-array in the end you could try empty(0) to start with, and then do hstack((A, your_scalar)) or something like that. Yeah, that works - odd, I thought concatenate((a,b),0) == hstack((a,b)) Thanks /David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
David Koch schrieb: > > In Python, I tried: > > A = empty((0,0)) > while > A = concatenate((A, array([someScalarValue])), 1) > end > > which returns an error since the shape of the empty A does not match the > vector I want to concatenate with. Any way to get around this without > having to use some obscure "if exist A" clause. I am still stuck in my > Matlab ways - hoping to get rid of them though. If you want a 1d-array in the end you could try empty(0) to start with, and then do hstack((A, your_scalar)) or something like that. hth, sven ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
David Koch wrote: > Hi, > > so one thing I came across now is the following, very simple: > > Matlab: > A = [] > while >A = [A some_scalar_value] > end > > > In Python, I tried: > > A = empty((0,0)) > while >A = concatenate((A, array([someScalarValue])), 1) > end > > which returns an error since the shape of the empty A does not match the > vector I want to concatenate with. Any way to get around this without > having > to use some obscure "if exist A" clause. I am still stuck in my Matlab ways > - hoping to get rid of them though. a = empty( (0,) ) ... I would bet that using a regular Python list would be faster in this case than growing the numpy array. a = [] while ... a.append( scalar ) a = array( a ) r. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
Hi, so one thing I came across now is the following, very simple: Matlab: A = [] while A = [A some_scalar_value] end In Python, I tried: A = empty((0,0)) while A = concatenate((A, array([someScalarValue])), 1) end which returns an error since the shape of the empty A does not match the vector I want to concatenate with. Any way to get around this without having to use some obscure "if exist A" clause. I am still stuck in my Matlab ways - hoping to get rid of them though. /Thanks, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
David Koch wrote:
> Hello,
>
> I am trying to translate some Matlab code to NumPy. I started reading
> the NumPy book and, yeah it's a very long read :-/ One thing I am
> completely confused about are the concpets of "basic" vs. "advanced"
> indexing. Are there some good examples out there where for the same
> piece of code - Matlab and NumPy are compared? I feel looking at those
> maybe more helpful to get me started than an in-depth study of the
> book. I already read "Numpy for Matlab users" and at a glance it
> doesn't seem to contain equivalents to all the common indexing
> operations you'd do in Matlab.
I have been there too ("long time" user of matlab, numpy convert for one
year now), and two links which help me a lot to have a direct mapping
between matlab and python at the beginning are:
http://www.scipy.org/NumPy_for_Matlab_Users
http://www.37mm.no/matlab-python-xref.html
Generally, except if you are doing a lot of linear algebra, and have to
write a lot things like dot(a, dot(b, dot(c, d))), my experience is that
you should avoid thinking the "matlab" way, and use nd array (that is
vector is rank 1, matrices are rank 2, etc...). It means that sometimes,
you will have to be more specific about some things, but make some other
things easier. For me (signal processing), once I switched to numpy way
of thinking, I found it easier to use nd arrays.
David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
Thank you everybody for your replies. Completely off-topic: I just read your sig Christoper: Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R Do you work with tsunami early-warning systems? I once had to give a presentation at Uni about the type of buoys that are deployed to detect tsunamis. Interesting stuff. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
David Koch wrote: > - Is it "pythonic" to initialize vectors to 2 dimensions so that > vec.shape == (len(vec), 1) instead of vec.shape == (len(vec),)? It depends what it means -- and this is not a question of Pythonic -- maybe Numpythonic? Numpy is an n-d array package -- NOT a matrix package (if you really want matrices, see the numpy matrix package). thus, a Vector can be represented as a (n,) shaped array a Matrix can be represented by a (n,m) shaped array etc... If what you want is a matrix that happens to have one column, you want a (n,1) array, if you want one row, you want a (1,n) array. In linear algebra terms, these are row and column vectors. In other math, a vector is (n,) in shape. Other than linear algebra, a reason to use 2-d "vectors" is for array broadcasting: >>> import numpy as N >>> x = N.arange(10).reshape(1,-1) # a "row" vector >>> y = N.arange(5).reshape(-1,1) # a "column" vector >>> x.shape (1, 10) >>> y.shape (5, 1) >>> z = x * y >>> z array([[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18], [ 0, 3, 6, 9, 12, 15, 18, 21, 24, 27], [ 0, 4, 8, 12, 16, 20, 24, 28, 32, 36]]) >>> ## note that you didn't' really need to reshape x, as a (n,) array is interpreted as a row vector for broadcasting purposes, but I like to do it for clarities sake. In MATLAB, there is no such thing as a vector, so you only have the last two options -- not the first. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On 3/5/07, David Koch <[EMAIL PROTECTED]> wrote: Hello, I am trying to translate some Matlab code to NumPy. I started reading the NumPy book and, yeah it's a very long read :-/ One thing I am completely confused about are the concpets of "basic" vs. "advanced" indexing. Are there some good examples out there where for the same piece of code - Matlab and NumPy are compared? I feel looking at those maybe more helpful to get me started than an in-depth study of the book. I already read "Numpy for Matlab users" and at a glance it doesn't seem to contain equivalents to all the common indexing operations you'd do in Matlab. Some concrete problems/questions I have right now: - Am I correct in assuming that all arrays have to be initialized to their final number of elements in NumPy (using empty/zero for instance)? Usually arrays are created from the data with their final dimensions set as part of the creation. You don't need to define an array and then fill in the values and it is best to avoid this where possible. Subarrays are best treated as views that require not copying of data, something that is missing from Matlab. - Given a matrix R, is there an equvialent to the Matlab operation R(:,j) = [] (which removes column j and "shrinks" the matrix? Look at delete. It is best to avoid such constructions when possible because they involve a data copy. - For (basic ?) indexing of an ndarray - is there any reason to prefer A[i,j] over A[i][j]? - Is it "pythonic" to initialize vectors to 2 dimensions so that vec.shape== (len(vec), 1) instead of vec.shape == (len(vec),)? There are two schools here, those who use matrices so that everything is 2d, and those who use arrays. One dimensional arrays will work fine for such things as matrix multiplication, so there is really no need for the second dimension. In any case, a faster way to implement your example is vec.reshape(-1,1) where the -1 fills in the dimension to use all of the array. matrix(vec) will return an 1xn matrix Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On 3/5/2007 2:13 PM, David Koch wrote: > - Given a matrix R, is there an equvialent to the Matlab > operation R(:,j) = [] (which removes column j and > "shrinks" the matrix? >>> help(numpy.delete) Help on function delete in module numpy.lib.function_base: delete(arr, obj, axis=None) Return a new array with sub-arrays along an axis deleted. Return a new array with the sub-arrays (i.e. rows or columns) deleted along the given axis as specified by obj obj may be a slice_object (s_[3:5:2]) or an integer or an array of integers indicated which sub-arrays to remove. If axis is None, then ravel the array first. Example: >>> arr = [[3,4,5], [1,2,3], [6,7,8]] >>> delete(arr, 1, 1) array([[3,5], [1,3], [6,8]) >>> delete(arr, 1, 0) array([[3,4,5], [6,7,8]]) hth, Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab -> NumPy translation and indexing
On 3/5/2007 2:13 PM, David Koch wrote: > - Am I correct in assuming that all arrays have to be initialized to > their final number of elements in NumPy (using empty/zero for instance)? You can also create an array from a Python list, data in a file, another array or a memory mapping. In these cases you don't need to know the size of the array in advance. But once created you cannot append elements to a NumPy array as you can with a Python list. NumPy arrays never call realloc due to the complication from view arrays. > - Given a matrix R, is there an equvialent to the Matlab operation > R(:,j) = [] (which removes column j and "shrinks" the matrix? Matlab does not have view arrays and no regard for memory consumption or fragmentation, so it can make a copy behind the scenes. In NumPy you have to be more explicit about what is done. def remove_column(a,j): idx = arange(0,a.shape[1]) idx = idx[idx != j] return a[:,idx] > - For (basic ?) indexing of an ndarray - is there any reason to prefer > A[i,j] over A[i][j]? The latter involves two function calls and creation of an intermediate object (a view array). > - Is it "pythonic" to initialize vectors to 2 dimensions so that > vec.shape == (len(vec), 1) instead of vec.shape == (len(vec),)? I have no idea. S.M. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Matlab -> NumPy translation and indexing
Hello, I am trying to translate some Matlab code to NumPy. I started reading the NumPy book and, yeah it's a very long read :-/ One thing I am completely confused about are the concpets of "basic" vs. "advanced" indexing. Are there some good examples out there where for the same piece of code - Matlab and NumPy are compared? I feel looking at those maybe more helpful to get me started than an in-depth study of the book. I already read "Numpy for Matlab users" and at a glance it doesn't seem to contain equivalents to all the common indexing operations you'd do in Matlab. Some concrete problems/questions I have right now: - Am I correct in assuming that all arrays have to be initialized to their final number of elements in NumPy (using empty/zero for instance)? - Given a matrix R, is there an equvialent to the Matlab operation R(:,j) = [] (which removes column j and "shrinks" the matrix? - For (basic ?) indexing of an ndarray - is there any reason to prefer A[i,j] over A[i][j]? - Is it "pythonic" to initialize vectors to 2 dimensions so that vec.shape== (len(vec), 1) instead of vec.shape == (len(vec),)? Alright, Thanks for now, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
