Re: [Numpy-discussion] replace voids in 2d dat with nearest neighbour value
Hi, I have a big 2 column data file where the data are repeated either 5 or 6 times. Are there any quick way to remove the duplicated data? Thanks Frank _ Rediscover Hotmail®: Get e-mail storage that grows with you. http://windowslive.com/RediscoverHotmail?ocid=TXT_TAGLM_WL_HM_Rediscover_Storage1_042009___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] how to delete the duplicated data in numpy?
Hi, I have just noticed that I did not change the title of my question. So I resend it out. Sorry for the mistake. Here is my question. I have a big 2 column data file where the data are repeated either 5 or 6 times. Are there any quick way to remove the duplicated data? Thanks Frank Rediscover Hotmail®: Get e-mail storage that grows with you. Check it out. _ Windows Live™: Keep your life in sync. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_allup_1a_explore_042009___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] help on fast slicing on a grid
I have created a test example for the question using for loop and hope someone
can help me to get fast solution. My data set is about 200 data.
However, I have the problem to run the code, the Out[i]=cnstl[j] line gives me
error says:
In [107]:
Out[0]=cnstl[0]---TypeError
Traceback (most recent call last)
C:\Frank_share\qamslicer.py in module() 1 2 3 4 5
TypeError: can't convert complex to float; use abs(z)
In [108]: cnstl.dtypeOut[108]: dtype('complex128')
I do not know why that my data is complex128 already. Can anyone help to figure
why?
Thanks
Frank
from numpy import *a =
arange(-15,16,2)cnstl=a.reshape(16,1)+1j*acnstl=cnstl.reshape(256,1)
X = array([1.4 + 1j*2.7, -4.9 + 1j*8.3])
Out = array(X)error =array(X)for i in xrange(2):for j in xrange(256):
a0 = real(X[i]) (real(cnstl[j])+1)a1 = real(X[i])
(real(cnstl[j])-1)a2 = imag(X[i]) (imag(cnstl[j])-1)a3 =
imag(X[i]) (imag(cnstl[j])+1) if (a0 a1 a2 a3):Out[i] =
cnstl[j]error[i] = X[i] - cnstl[j]
From: f...@hotmail.comto: numpy-discuss...@scipy.orgsubject: RE:
[Numpy-discussion] help on fast slicing on a gridDate: Wed, 28 Jan 2009
23:28:47 -0700
Hi, Bob, Thanks for your help. I am sorry for my type error. qam array is the
X array in my example. cntl is a complex array contains the point (x,y) axises.
I will try to make a workable example. Also I will try to find out the
zeros_like function. However, I guess that zeros_like(X) will create an array
the same size as X. It it is. Then the two line Out=X and error=X should be
Out=zeros_like(X) and error=zeros(X). Also, can where command handel the logic
command? aa = np.where((real(X)real(cnstl[j])+1) (real(X)real(cnstl[j])-1)
(imag(X)imag(cnstl[j])+1) (imag(X)imag(cnstl[j]-1)) For example,
cntl[j]=3+1j*5, then the where command is the same as: aa =
np.where((real(X)4) (real(X)2 ) (imag(X)6) (imag(X)4)) Thanks Frank
Date: Thu, 29 Jan 2009 00:15:48 -0600 From: robert.k...@gmail.com To:
numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] help on fast
slicing on a grid On Thu, Jan 29, 2009 at 00:09, frank wang
f...@hotmail.com wrote: Here is the for loop that I am think about. Also,
I do not know whether the where commands can handle the complicated logic.
The where command basically find the data in the square around the point
cnstl[j]. cnstl is a 2D array from your previous description. Let the
data array is qam with size N I don't see qam anywhere. Did you mean X?
Out = X error = X Don't you want something like zeros_like(X) for these?
for i in arange(N): for j in arange(L): aa =
np.where((real(X)real(cnstl[j])+1) (real(X)real(cnstl[j])-1)
(imag(X)imag(cnstl[j])+1) (imag(X)imag(cnstl[j]-1)) Out[aa]=cnstl[j]
error[aa]=abs(X)**2 - abs(cnstl[j])**2 I'm still confused. Can you show me
a complete, working script with possibly fake data? -- Robert Kern I
have come to believe that the whole world is an enigma, a harmless enigma that
is made terrible by our own mad attempt to interpret it as though it had an
underlying truth. -- Umberto Eco
___ Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Windows Live™: E-mail. Chat. Share. Get more ways to connect. Check it out.
_
Hotmail® goes where you go. On a PC, on the Web, on your phone.
http://www.windowslive-hotmail.com/learnmore/versatility.aspx#mobile?ocid=TXT_TAGHM_WL_HM_versatility_121208
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] help on fast slicing on a grid
Hi, Bob, Thanks. This solution works great. It really helps me a lot. Frank Date: Fri, 30 Jan 2009 23:08:35 -0600 From: robert.k...@gmail.com To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] help on fast slicing on a grid On Fri, Jan 30, 2009 at 22:41, frank wang f...@hotmail.com wrote: Thanks for the correction. I will learn the ravel() function since I do not know it. Moving from Matlab world into python is tricky sometime. Your output In [22]: out Out[22]: array([ 1.+3.j, -5.+9.j]) In [23]: error Out[23]: array([ 0.4-0.3j, 0.1-0.7j]) are correct answer. However, if my data set is large, this solution takes long time to run. Are there any python/numpy magic to speed it up? from numpy import * a = arange(-15,16,2) cnstl = a[:,newaxis] + 1j*a cnstl = cnstl.ravel() X = array([1.4 + 1j*2.7, -3.9 + 1j*8.3]) out = around((X + 1+1j) / 2.0) * 2.0 - (1+1j) error = X - out print out print error -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ Windows Live™ Hotmail®:…more than just e-mail. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_explore_012009___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] help on fast slicing on a grid
Hi, I have to buidl a grid with 256 point by the command: a = arange(-15,16,2) L = len(a) cnstl = a.reshape(L,1)+1j*a My problem is that I have a big data array that contains the data round the points in cnstl. I want to slice the point to the closest cnstl point and also compute the error. The condition is in the middle of the two point in x and y axis. I can do it in a for loop. Since Python and numpy have a lot of magic, I want to find an efficient way to do. This problem arise from QAM 256 modulation. Thanks Frank _ Windows Live™ Hotmail®:…more than just e-mail. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_hm_justgotbetter_explore_012009___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] help on fast slicing on a grid
Here is the for loop that I am think about. Also, I do not know whether the where commands can handle the complicated logic. The where command basically find the data in the square around the point cnstl[j]. Let the data array is qam with size N Out = X error = X for i in arange(N): for j in arange(L): aa = np.where((real(X)real(cnstl[j])+1) (real(X)real(cnstl[j])-1) (imag(X)imag(cnstl[j])+1) (imag(X)imag(cnstl[j]-1)) Out[aa]=cnstl[j] error[aa]=abs(X)**2 - abs(cnstl[j])**2 Thanks Frank Date: Wed, 28 Jan 2009 23:57:16 -0600 From: robert.k...@gmail.com To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] help on fast slicing on a grid On Wed, Jan 28, 2009 at 23:52, frank wang f...@hotmail.com wrote: Hi, I have to buidl a grid with 256 point by the command: a = arange(-15,16,2) L = len(a) cnstl = a.reshape(L,1)+1j*a My problem is that I have a big data array that contains the data round the points in cnstl. I want to slice the point to the closest cnstl point and also compute the error. The condition is in the middle of the two point in x and y axis. I can do it in a for loop. Since Python and numpy have a lot of magic, I want to find an efficient way to do. This problem arise from QAM 256 modulation. Can you show us the for loop? I'm not really sure what you want to compute. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ Windows Live™: E-mail. Chat. Share. Get more ways to connect. http://windowslive.com/howitworks?ocid=TXT_TAGLM_WL_t2_allup_howitworks_012009___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] help on fast slicing on a grid
Hi, Bob, Thanks for your help. I am sorry for my type error. qam array is the X array in my example. cntl is a complex array contains the point (x,y) axises. I will try to make a workable example. Also I will try to find out the zeros_like function. However, I guess that zeros_like(X) will create an array the same size as X. It it is. Then the two line Out=X and error=X should be Out=zeros_like(X) and error=zeros(X). Also, can where command handel the logic command? aa = np.where((real(X)real(cnstl[j])+1) (real(X)real(cnstl[j])-1) (imag(X)imag(cnstl[j])+1) (imag(X)imag(cnstl[j]-1)) For example, cntl[j]=3+1j*5, then the where command is the same as: aa = np.where((real(X)4) (real(X)2 ) (imag(X)6) (imag(X)4)) Thanks Frank Date: Thu, 29 Jan 2009 00:15:48 -0600 From: robert.k...@gmail.com To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] help on fast slicing on a grid On Thu, Jan 29, 2009 at 00:09, frank wang f...@hotmail.com wrote: Here is the for loop that I am think about. Also, I do not know whether the where commands can handle the complicated logic. The where command basically find the data in the square around the point cnstl[j]. cnstl is a 2D array from your previous description. Let the data array is qam with size N I don't see qam anywhere. Did you mean X? Out = X error = X Don't you want something like zeros_like(X) for these? for i in arange(N): for j in arange(L): aa = np.where((real(X)real(cnstl[j])+1) (real(X)real(cnstl[j])-1) (imag(X)imag(cnstl[j])+1) (imag(X)imag(cnstl[j]-1)) Out[aa]=cnstl[j] error[aa]=abs(X)**2 - abs(cnstl[j])**2 I'm still confused. Can you show me a complete, working script with possibly fake data? -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ Windows Live™: E-mail. Chat. Share. Get more ways to connect. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_allup_explore_012009___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] how do I delete unused matrix to save the memory?
I am running in ipython. Now I do not have the problem anymore. %reset commands is a good solution. Thanks Frank Date: Tue, 9 Dec 2008 21:03:00 -0600 From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] how do I delete unused matrix to save the memory? On Mon, Dec 8, 2008 at 19:15, frank wang [EMAIL PROTECTED] wrote: Hi, I have a program with some variables consume a lot of memory. The first time I run it, it is fine. The second time I run it, I will get MemoryError. If I close the ipython and reopen it again, then I can run the program once. I am looking for a command to delete the intermediate variable once it is not used to save memory like in matlab clear command. How are you running this program? Be aware that IPython may be holding on to objects and preventing them from being deallocated. For example: In [7]: !cat memtest.py class A(object): def __del__(self): print 'Deleting %r' % self a = A() In [8]: %run memtest.py In [9]: %run memtest.py In [10]: %run memtest.py In [11]: del a In [12]: Do you really want to exit ([y]/n)? $ python memtest.py Deleting __main__.A object at 0x915ab0 You can remove some of these references with %reset and maybe a gc.collect() for good measure. In [1]: %run memtest In [2]: %run memtest In [3]: %run memtest In [4]: %reset Once deleted, variables cannot be recovered. Proceed (y/[n])? y Deleting __main__.A object at 0xf3e950 Deleting __main__.A object at 0xf3e6d0 Deleting __main__.A object at 0xf3e930 -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ Send e-mail faster without improving your typing skills. http://windowslive.com/Explore/hotmail?ocid=TXT_TAGLM_WL_hotmail_acq_speed_122008___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Importance of order when summing values in anarray
On my two systems with Intel Core2 DUO, finfo(float128) gives me the nameerro, NameError: name 'float128' is not defined. Why? Thanks Frank Date: Tue, 9 Dec 2008 21:10:32 -0600 From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] Importance of order when summing values in anarray On Tue, Dec 9, 2008 at 21:01, Charles R Harris [EMAIL PROTECTED] wrote:On Tue, Dec 9, 2008 at 1:40 PM, Robert Kern [EMAIL PROTECTED] wrote: On Tue, Dec 9, 2008 at 09:51, Nadav Horesh [EMAIL PROTECTED] wrote: As much as I know float128 are in fact 80 bits (64 mantissa + 16 exponent) so the precision is 18-19 digits (not 34) float128 should be 128 bits wide. If it's not on your platform, please let us know as that is a bug in your build. I think he means the actual precision is the ieee extended precision, the number just happens to be stored into larger chunks of memory for alignment purposes. Ah, that's good to know. Yes, float128 on my Intel Mac behaves this way. In [12]: f = finfo(float128) In [13]: f.nmant Out[13]: 63 In [14]: f.nexp Out[14]: 15 -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ You live life online. So we put Windows on the web. http://clk.atdmt.com/MRT/go/127032869/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] how to create a matrix based on a vector?
Hi, I want to create a matrix based on a vector. It is difficult to describe the issue for me in english. Here is an example. Suppose I have an array([3, 6, 8, 12]), I want to create a range based on each element. In this exampe, let us say want to create 4 number with step 2, so I will have [3, 6, 8, 12 5, 8, 10,14 7, 10,12,16 9, 12,14,18] It is a 4 by 4 maxtric in this example. My original array is quite large. but the range I want to create around the number is not big, it is about 30. Does anyone know how to do this efficiently? Thanks Frank _ Send e-mail faster without improving your typing skills. http://windowslive.com/Explore/hotmail?ocid=TXT_TAGLM_WL_hotmail_acq_speed_122008___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] how to create a matrix based on a vector?
I got a lof of help from the experts in this forum. I resitsted to send a thank you reply for fearing spaming the forum. This time I really want to let the people know that I am really appreciate the great help I got. Please let me know if a simple thank you message is not appropriate in this forum. Numpy makes Pyhton a great tools for processing signal. Thank you very much. Frank Date: Mon, 8 Dec 2008 11:30:31 -0600 From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] how to create a matrix based on a vector? On Mon, Dec 8, 2008 at 11:27, frank wang [EMAIL PROTECTED] wrote: Hi, I want to create a matrix based on a vector. It is difficult to describe the issue for me in english. Here is an example. Suppose I have an array([3, 6, 8, 12]), I want to create a range based on each element. In this exampe, let us say want to create 4 number with step 2, so I will have [3, 6, 8, 12 5, 8, 10,14 7, 10,12,16 9, 12,14,18] It is a 4 by 4 maxtric in this example. My original array is quite large. but the range I want to create around the number is not big, it is about 30. Does anyone know how to do this efficiently? In [1]: from numpy import * In [2]: a = array([3, 6, 8, 12]) In [4]: b = arange(0, 4*2, 2)[:,newaxis] In [5]: a+b Out[5]: array([[ 3, 6, 8, 12], [ 5, 8, 10, 14], [ 7, 10, 12, 16], [ 9, 12, 14, 18]]) -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ Send e-mail faster without improving your typing skills. http://windowslive.com/Explore/hotmail?ocid=TXT_TAGLM_WL_hotmail_acq_speed_122008___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] how do I delete unused matrix to save the memory?
Hi, I have a program with some variables consume a lot of memory. The first time I run it, it is fine. The second time I run it, I will get MemoryError. If I close the ipython and reopen it again, then I can run the program once. I am looking for a command to delete the intermediate variable once it is not used to save memory like in matlab clear command. Thanks Frank _ Send e-mail faster without improving your typing skills. http://windowslive.com/Explore/hotmail?ocid=TXT_TAGLM_WL_hotmail_acq_speed_122008___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] fast way to convolve a 2d array with 1d filter
Hi, I need to convolve a 1d filter with 8 coefficients with a 2d array of the shape (6,7). I can use convolve to perform the operation for each row. This will involve a for loop with a counter 6. I wonder there is an fast way to do this in numpy without using for loop. Does anyone know how to do it? Thanks Frank _ Access your email online and on the go with Windows Live Hotmail. http://windowslive.com/Explore/Hotmail?ocid=TXT_TAGLM_WL_hotmail_acq_access_112008___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fast way to convolve a 2d array with 1d filter
This is what I thought to do. However, I am not sure whether this is a fast way to do it and also I want to find a more generous way to do it. I thought there may be a more elegant way to do it. Thanks Frank Date: Tue, 2 Dec 2008 07:42:27 +0200 From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] fast way to convolve a 2d array with 1d filter Hi Frank 2008/12/2 frank wang [EMAIL PROTECTED]: I need to convolve a 1d filter with 8 coefficients with a 2d array of the shape (6,7). I can use convolve to perform the operation for each row. This will involve a for loop with a counter 6. I wonder there is an fast way to do this in numpy without using for loop. Does anyone know how to do it? Since 6x7 is quite small, so you can afford this trick: a) Pad the 6,7 array to 6,14. b) Flatten the array c) Perform convolution d) Unflatten array e) Take out valid values Cheers Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ Get more done, have more fun, and stay more connected with Windows Mobile®. http://clk.atdmt.com/MRT/go/119642556/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] fill() function does not work.
Hi, My numpy is 1.2.1 and python is 2.5.2. In python, I did: from numpy import * x=array([1,2,3]) z=x.fill(x) print z None z should be filled with zero. I do not knwo why I got None. Can anyone help me on this? Thanks Frank _ Proud to be a PC? Show the world. Download the “I’m a PC” Messenger themepack now. hthttp://clk.atdmt.com/MRT/go/119642558/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Fast and efficient way to convert an array into binary
Hi, I have a large array and I want to convert it into a binary array. For exampe, y=array([1,2,3]), after the convertion I want the result array([0,0,0,1,0,0,1,0,0,0,1,1]). Each digit is converted into 4 bits in this example. In my real problem I want to convert each digit to 8 bits. My data is numpy.ndarray and the shape is, say, (1000,). Are there fast and efficient solution for this? Thanks Frank _ Color coding for safety: Windows Live Hotmail alerts you to suspicious email. http://windowslive.com/Explore/Hotmail?ocid=TXT_TAGLM_WL_hotmail_acq_safety_112008 ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Numpy Example List With Doc cannot be printed
Hi, From www.scipy.org web site, I tried to print the Numpy Example List with Doc. Even though I can read the document on the computer, but when I printed, except of the first few pages, all pages printed empty. Does anyone know the reason? Thanks Frank _ Stay up to date on your PC, the Web, and your mobile phone with Windows Live http://clk.atdmt.com/MRT/go/119462413/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy and MKL, update
Hi, Can you provide a working example to build Numpy with MKL in window and linux? The reason I am thinking to build the system is that I need to make the speed match with matlab. Thanks frank Date: Thu, 13 Nov 2008 11:39:42 +0100 From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Subject: [Numpy-discussion] Numpy and MKL, update Hi, I got an answer from the Intel Support about the issues between MKL and Numpy (and Matlab and ...). To use MKL with Numpy, we must know explicitely use the static MKL libraries (libmkl_intel_lp64.a, libmkl_intel_thread.a and libmkl_core.a). The same applies for Scipy and every other Python module you build with the MKL. Matthieu -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ See how Windows® connects the people, information, and fun that are part of your life http://clk.atdmt.com/MRT/go/119463819/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] signal processing filter operation in numpy
Hi, I need to perform iir filter operation using numpy and could not googled any useful info for this. Are there filter operation similar the matlab filter funciton in Numpy? Thanks Frank _ Color coding for safety: Windows Live Hotmail alerts you to suspicious email. http://windowslive.com/Explore/Hotmail?ocid=TXT_TAGLM_WL_hotmail_acq_safety_112008___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] (no subject)
Hi, In my work, I want to implement a fir filter with an input array. Since performing the filter on each input sample is slow, are there fast way to perform the fir filter operation? Are there ways to convert input into an array and perform the array multipication? Thanks Frank _ When your life is on the go—take your life with you. http://clk.atdmt.com/MRT/go/115298558/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] help to speed up the python code
Hi, Bob, The problem is that I want to resample my data with another sampling rate. the two rates is very close. I use the formula: s(t)=sum(a_k*sinc(t-kTs)). the new sampling rate is Ts', so I have s(nTs')=sum(a_k*sinc(nTs'-kTs)). The sum index k is over the (-P, P), Centered at n. The n is start from zero. THe code is using two for loops and it is slow. The length of s(nTs) is very long, so it takes quite long time to do it. Thanks Frank Date: Sun, 26 Oct 2008 22:45:42 -0500 From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] help to speed up the python code On Fri, Oct 24, 2008 at 11:30, frank wang [EMAIL PROTECTED] wrote: Hi, I have to send this request second time since my first message contains the attached data file which is too big and was blocked by the system. So this time I will not attach the data file. I have converted a matlab function to python using numpy. both matlab and python run slow. I know that numpy has a lot of features, so I hope some experts can help me to speed up the code. Can you describe in higher level terms what you are trying to do? I'm having trouble following the code. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ Stay organized with simple drag and drop from Windows Live Hotmail. http://windowslive.com/Explore/hotmail?ocid=TXT_TAGLM_WL_hotmail_102008___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] help to speed up the python code
Hi, I have to send this request second time since my first message contains the attached data file which is too big and was blocked by the system. So this time I will not attach the data file. I have converted a matlab function to python using numpy. both matlab and python run slow. I know that numpy has a lot of features, so I hope some experts can help me to speed up the code. Here is how I run the code: upsample.upsample(cdata,4*1024*401.0/812.0,2560.0,'r') Where cdata is about 7 complex data. Thanks Frank from numpy import zeros,ceil,pi,arange,concatenate,sincfrom pylab import plot,clf,show,figure, psd, grid,xlabel, figureimport pdbdef upsample(input,Fs_old,Fsamp,filt_type): Perform resampling the input data from rate Fs to Fsamp Note:y=zeros((N)) shape is (N,). y=zeros((N,1)) shape is (N,1). Example of how to read a two columns floating data file created by Matlab. d=fromfile(filename,dtype='float',count=-1,sep=' ') x=len(d) data=d.reshape([x/2,2]) Ts=1.0/Fs_old Tsamp=1.0/Fsamp Fw=600.0 L=len(input) N=ceil(Fsamp/Fs_old*L) y=zeros((N),dtype='float64') C=pi*Fw t0=arange(0,Ts,Tsamp) #print t0 P = 16 input=concatenate((zeros((P)),input,zeros((P))),1) #print input out = 0 for mm in arange(P+1): tt=t0-mm*Ts out=out+input[P+mm]*sinc(Fw*tt) #print tt #print out\n #print out y[0:len(t0)]=out #print y B=len(t0) for m in arange(P+2,L+P+1): delta=Tsamp-(Ts-t0[-1]) t1=arange(delta,Ts,Tsamp) out=0 for mm in arange(-P,P+1): tt=(m-1-P)*Ts+t1-(mm+m-(P+2)+1)*Ts out=out+input[m+mm-1]*sinc(Fw*tt) y[B:B+len(t1)]=out t0=t1 B=B+len(t1) clf() figure(4) psd(y,256,Fs=25.6) #show() _ You live life beyond your PC. So now Windows goes beyond your PC. http://clk.atdmt.com/MRT/go/115298556/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Help to process a large data file
Thans David and Chris for providing the nice solution.
Both method works gread. I could not tell the speed difference between the two
solutions. My data size is 1048577 lines.
I did not try the second solution from Chris since it is too slow as Chris
stated.
Frank
Date: Thu, 2 Oct 2008 17:43:37 +0200 From: [EMAIL PROTECTED] To:
numpy-discussion@scipy.org CC: [EMAIL PROTECTED] Subject: Re:
[Numpy-discussion] Help to process a large data file Frank, I would
imagine that you cannot get a much better performance in python than this,
which avoids string conversions: c = [] count = 0 for line in
open('foo'): if line == '1 1\n': c.append(count) count = 0 else: if '1'
in line: count += 1 One could do some numpy trick like: a =
np.loadtxt('foo',dtype=int) a = np.sum(a,axis=1) # Add the two columns
horizontally b = np.where(a==2)[0] # Find with sum == 2 (1 + 1) count = []
for i,j in zip(b[:-1],b[1:]): count.append( a[i+1:j].sum() ) # Calculate
number of lines with 1 but on my machine the numpy version takes about 20
sec for a 'foo' file of 2,500,000 lines versus 1.2 sec for the pure python
version... As a side note, if i replace line == '1 1\n' with
line.startswith('1 1'), the pure python version goes up to 1.8 sec...
Isn't this a bit weird, i'd think startswith() should be faster...
Chris On Wed, Oct 01, 2008 at 07:27:27PM -0600, frank wang wrote:
Hi,I have a large data file which contains 2 columns of data. The two
columns only have zero and one. Now I want to cound how many one in
between if both columns are one. For example, if my data is:1 0 0
0 1 1 0 0 0 1 x 0 1 x 0 0 0 1 x 1 1 0 0 0 1 x 0
1 x 1 1Then my count will be 3 and 2 (the numbers with x).
Are there an efficient way to do this? My data file is pretty big.
ThanksFrank ___
Numpy-discussion mailing list Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
_
See how Windows connects the people, information, and fun that are part of your
life.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093175mrt/direct/01/___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Help to process a large data file
Hi, I have a large data file which contains 2 columns of data. The two columns only have zero and one. Now I want to cound how many one in between if both columns are one. For example, if my data is: 1 0 0 0 1 1 0 0 0 1x 0 1x 0 0 0 1x 1 1 0 0 0 1x 0 1x 1 1 Then my count will be 3 and 2 (the numbers with x). Are there an efficient way to do this? My data file is pretty big. Thanks Frank _ See how Windows connects the people, information, and fun that are part of your life. http://clk.atdmt.com/MRT/go/msnnkwxp1020093175mrt/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Are there command similar as Matlab find command?
Hi, I am trying to find a command in numpy or python that perform similar function as Matlab find command. It will return the indexes of array that satisfy a condition. So far I have not found anything. Thanks Frank Date: Thu, 25 Sep 2008 09:51:46 +0200 From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] loadtxt error 2008/9/24 frank wang [EMAIL PROTECTED]: Thank you very much for all of you. I have downloaded the binary version 1.2rc and it fixed the problem. My special thanks to the person who created the window binary version for users who do not know or do not have the capacity to build the numpy from source. That would be David Cournapeau -- one of the few brave enough to venture there. Cheers Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ Stay up to date on your PC, the Web, and your mobile phone with Windows Live. http://clk.atdmt.com/MRT/go/msnnkwxp1020093185mrt/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Are there command similar as Matlab find command?
Thanks for the help.
It seems that the where command has problem when I tried to run it in the debug
mode. It does not return any thing such as:
(Pdb) aa=array([1,2,3,4]
(Pdb) where(aa2)
stdin(1)module() c:\dhg\docsis\lab_test\parseadc.py(70)parsempeg()-
bb=array(fid).astype('int')
(Pdb)
It does not return any result.
Frank Date: Mon, 29 Sep 2008 16:48:23 -0500 From: [EMAIL PROTECTED] To:
numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] Are there command
similar as Matlab find command? frank wang wrote: Hi,I am trying
to find a command in numpy or python that perform similar function as
Matlab find command. It will return the indexes of array that satisfy a
condition. So far I have not found anything. There are several ways to do
this, but what are you trying to do? Non-zero on the boolean array resulting
from the condition is the most direct way: (a30).nonzero() where(a30)
This returns a tuple of indices of length nd, where nd is the number of
dimensions of a. (i.e. for 1-d case you need to extract the first element of
the tuple to get the indices you want). But, if you are going to use these
indices to access elements of the array, there are better ways to do that:
a[a30] compress(a30, a) etc. -Travis
___ Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
_
Want to do more with Windows Live? Learn “10 hidden secrets” from Jamie.
http://windowslive.com/connect/post/jamiethomson.spaces.live.com-Blog-cns!550F681DAD532637!5295.entry?ocid=TXT_TAGLM_WL_domore_092008___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Are there command similar as Matlab find command?
Hi, Robert,
Thank you very much for clarification of this.
I really do not know the difference of debug mode and the pdb debugger. To me,
it seems that pdb is only way to debug the python code. How do the expert of
numpy/python debug their code? Are there any more efficient way to debug the
code in python world? I used to use matlab which comes with a nice debugger.
But now I really want to try the open source software and I found python/numpy
is a nice tool. The only lagging piece of python/numpy comparing with matlab is
the powerful debuggign capability.
Onece again, I really appreciate the help I got from the forum.
Frank Date: Mon, 29 Sep 2008 20:22:14 -0500 From: [EMAIL PROTECTED] To:
numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] Are there command
similar as Matlab find command? On Mon, Sep 29, 2008 at 20:16, frank wang
[EMAIL PROTECTED] wrote: Thanks for the help. It seems that the
where command has problem when I tried to run it in the debug mode. It does
not return any thing such as: (Pdb) aa=array([1,2,3,4] (Pdb)
where(aa2) stdin(1)module()
c:\dhg\docsis\lab_test\parseadc.py(70)parsempeg() -
bb=array(fid).astype('int') (Pdb) It does not return any result.
It's worth noting that that is not a debug mode of the interpreter; it's the
pdb debugger, a separate piece of software which is used in an entirely
different manner. Specifically, pdb has a where command which is what you
are getting here. (Pdb) ?where w(here) Print a stack trace, with the most
recent frame at the bottom. An arrow indicates the current frame, which
determines the context of most commands. 'bt' is an alias for this command.
-- Robert Kern I have come to believe that the whole world is an enigma,
a harmless enigma that is made terrible by our own mad attempt to interpret it
as though it had an underlying truth. -- Umberto Eco
___ Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
_
Stay up to date on your PC, the Web, and your mobile phone with Windows Live.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093185mrt/direct/01/___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] loadtxt error
Thank you very much for all of you. I have downloaded the binary version 1.2rc and it fixed the problem. My special thanks to the person who created the window binary version for users who do not know or do not have the capacity to build the numpy from source. Frank Date: Wed, 24 Sep 2008 17:10:45 +0200 From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] loadtxt error 2008/9/24 Joshua Lippai [EMAIL PROTECTED]: And the version would be displayed on screen. Bear in mind that unlike the release, which installs via an installer file you double click, you will have to compile numpy from the downloaded source code yourself. Detailed instructions for doing this (and also installing SciPy from source) are available here: In this case, you can simply grab the io.py file and replace your version. Good luck, Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ See how Windows connects the people, information, and fun that are part of your life. http://clk.atdmt.com/MRT/go/msnnkwxp1020093175mrt/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance of the numpy
Hi, All, I am using ipython with --pylab flag. ipython loads the numpy into the workspace, so I do not know abs is from python or numpy. The weird thing is if I execute the code line by line, I do not have any speed problem. But when I combine them together into one command, it slowdonws the computer significantly. From my understanding, using the modulename.functionname will slow down the python performance. For a big simulation, it may not be a good idear. Are there any suggestion for the matlab uses who want to use numpy/scipy how to setup their working environment? Thanks Frank Date: Wed, 24 Sep 2008 15:37:03 -0700 From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] performance of the numpy Nadav Horesh wrote: You should use absolute (a ufunc) and not abs (internal python function):plot(absolute(fft(b))) another reason why import * is a bad idea: import numpy as np import pylab as plot #(what is the convention for this now?) pylab.plot(np.absolute(np.fft(b))) yes, it's more typing, but you'll never get confused as to what module functions come from. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/ORR/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion _ See how Windows connects the people, information, and fun that are part of your life. http://clk.atdmt.com/MRT/go/msnnkwxp1020093175mrt/direct/01/___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] loadtxt error
My numpy version is numpy-1.1.1-win32-superpack-python2.5.exe. It is the newest release. By the way, in Python, how can I check numpy version? Thanks Frank Date: Tue, 23 Sep 2008 08:42:00 -0400From: [EMAIL PROTECTED]: [EMAIL PROTECTED]: Re: [Numpy-discussion] loadtxt error This bug has been fixed in the trunk a couple of weeks ago. Cheers, David On Mon, Sep 22, 2008 at 8:10 PM, Pierre GM [EMAIL PROTECTED] wrote: On Monday 22 September 2008 19:56:47 frank wang wrote: This error is caused that the usecols is a tuple and it does not have find command. I do not know how to fix this problem.Try to use a list instead of a tuple as a quick-fix.Anyway, Frank, you should try to give us the version of numpy you're using.Obviously, it's not the latest. ___Numpy-discussion mailing [EMAIL PROTECTED]://projects.scipy.org/mailman/listinfo/numpy-discussion _ Get more out of the Web. Learn 10 hidden secrets of Windows Live. http://windowslive.com/connect/post/jamiethomson.spaces.live.com-Blog-cns!550F681DAD532637!5295.entry?ocid=TXT_TAGLM_WL_domore_092008___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] loadtxt error
My numpy version is 1.1.1. I just downloaded and installed. It is the same
result. Also, when I use the list, I got similar error said list does not have
the find command.
--- 14 fid=loadtxt(fname,comments='',dtype='|S4',converters={cols:lambda
s:int(s,16)},usecols=[cols]) 15 16
#fid=loadtxt('ww36_5adcoutputsin45mhznotuner-0dbm_mux_adc_ddc_rmR.csv',comments='',dtype='string',usecols=(0,))
C:\Python25\lib\site-packages\numpy\lib\io.pyc in loadtxt(fname, dtype,
comments, delimiter, converters, skiprows, usecols, unpack)337 for i,
conv in (user_converters or {}).iteritems():338 if usecols:-- 339
i = usecols.find(i)340 converters[i] = conv341
AttributeError: 'list' object has no attribute 'find'
c:\python25\lib\site-packages\numpy\lib\io.py(339)loadtxt()338 if
usecols:-- 339 i = usecols.find(i)340 converters[i] =
conv
Thanks
Frank From: [EMAIL PROTECTED] To: numpy-discussion@scipy.org Date: Mon, 22
Sep 2008 20:10:11 -0400 Subject: Re: [Numpy-discussion] loadtxt error On
Monday 22 September 2008 19:56:47 frank wang wrote: This error is caused
that the usecols is a tuple and it does not have find command. I do not know
how to fix this problem. Try to use a list instead of a tuple as a
quick-fix. Anyway, Frank, you should try to give us the version of numpy
you're using. Obviously, it's not the latest.
___ Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
_
See how Windows connects the people, information, and fun that are part of your
life.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093175mrt/direct/01/___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] loadtxt error
Hi, All,
I am struggling to make the loadtxt works. In my file, I have several colunms
of data, say I have two. When I use the following command to load the data,
fid=loadtxt('filename.csv',comments='',dtype='|S4',converters={0:lambda
s:int(s,16)})
It will load an array has two columns. However, I only want to load one column
to save the memory. If I type
fid=loadtxt('filename.csv',comments='',dtype='|S4',converters={0:lambda
s:int(s,16)},usecols=(1,))
I got error:
337 for i, conv in (user_converters or {}).iteritems():338
if usecols:-- 339 i = usecols.find(i)340 converters[i]
= conv341
AttributeError: 'tuple' object has no attribute 'find'
c:\python25\lib\site-packages\numpy\lib\io.py(339)loadtxt()338 if
usecols:-- 339 i = usecols.find(i)340 converters[i] =
conv
Can someone help to solve this problem? Or is this a bug in loadtxt?
Here is a test file with contains:
ADCOUT Nyquist29F 23529F 397
Thanks
Frank
_
See how Windows Mobile brings your life together—at home, work, or on the go.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093182mrt/direct/01/___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] A bug in loadtxt and how to convert a string array (hex data) to decimal?
Hi, All,
I have found a bug in the loadtxt function. Here is the example. The file name
is test.txt and contains:
Thist is test
3FF 3fE
3Ef 3e8
3Df 3d9
3cF 3c7
In the Python 2.5.2, I type:
test=loadtxt('test.txt',comments='',dtype='string',converters={0:lambda
s:int(s,16)})
test will contain
array([['102', '3fE'],
['100', '3e8'],
['991', '3d9'],
['975', '3c7']],
dtype='|S3')
The first two values 102 and 100 are wrong.
The reason I am doing this because that I have to process a large amount of
data from a file. The data is in hex format. This is only way I found that I
can efficiently convert the hex to decimal.
Anyone has a good solution?
Thanks
Frank
_
Get more out of the Web. Learn 10 hidden secrets of Windows Live.
http://windowslive.com/connect/post/jamiethomson.spaces.live.com-Blog-cns!550F681DAD532637!5295.entry?ocid=TXT_TAGLM_WL_domore_092008___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A bug in loadtxt and how to convert a string array (hex data) to decimal?
Hi, Ryan,
Thank you very much. It solves my problem. I have struggled with this for long
time.
Frank Date: Thu, 18 Sep 2008 16:39:47 -0500 From: [EMAIL PROTECTED] To:
numpy-discussion@scipy.org Subject: Re: [Numpy-discussion] A bug in loadtxt
and how to convert a string array (hex data) to decimal? frank wang wrote:
Hi, All,I have found a bug in the loadtxt
function. Here is the example. The file name is test.txt and contains:
Thist is test 3FF 3fE 3Ef 3e8 3Df 3d9
3cF 3c7In the Python 2.5.2, I type:
test=loadtxt('test.txt',comments='',dtype='string',converters={0:lambda
s:int(s,16)})test will contain
array([['102', '3fE'], ['100', '3e8'], ['991', '3d9'],
['975', '3c7']], dtype='|S3')It's because of how
numpy handles strings arrays (which I admit I don't understand very well.)
Basically, it's converting the numbers properly, but truncating them to 3
characters. Try this, which just forces it to expand to strings 4 characters
wide: test=loadtxt('test.txt',comments='',dtype='|S4',converters={0:lambda
s:int(s,16)}) HTH, Ryan -- Ryan May Graduate Research Assistant
School of Meteorology University of Oklahoma
___ Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
_
See how Windows connects the people, information, and fun that are part of your
life.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093175mrt/direct/01/___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
