Thanks so much for your help on the '*' confusion. It makes sense now. Thanks, Frank
On Fri, Jul 25, 2008 at 3:57 PM, <[EMAIL PROTECTED]> wrote: > Send Numpy-discussion mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://projects.scipy.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > [EMAIL PROTECTED] > > You can reach the person managing the list at > [EMAIL PROTECTED] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Numpy-discussion digest..." > > Today's Topics: > > 1. Re: [Cdat-discussion] Arrays containing NaNs (Charles Doutriaux) > 2. numpy-1.1.1rc2 Mac binary - Please Test. (Christopher Burns) > 3. Re: numpy-1.1.1rc2 Mac binary - Please Test. (Alan McIntyre) > 4. curious problem with SVD (Frank Lagor) > 5. Re: curious problem with SVD (Keith Goodman) > 6. Re: curious problem with SVD (Keith Goodman) > 7. Re: curious problem with SVD (Robert Kern) > 8. Re: curious problem with SVD (Arnar Flatberg) > > > ---------- Forwarded message ---------- > From: Charles Doutriaux <[EMAIL PROTECTED]> > To: Pierre GM <[EMAIL PROTECTED]> > Date: Fri, 25 Jul 2008 11:43:41 -0700 > Subject: Re: [Numpy-discussion] [Cdat-discussion] Arrays containing NaNs > Hi Pierre, > > Thanks for the answer, I'm ccing cdat's discussion list. > > It makes sense, that's also the way we develop things here NEVER assume > what the user is going to do with the data BUT give the user the necessary > tools to do what you're assuming he/she wants to do (as simple as possible) > > Thanks again for the answer. > > C. > > > Pierre GM wrote: > >> Oh, I guess this one's for me... >> >> On Thursday 01 January 1970 04:21:03 Charles Doutriaux wrote: >> >> >> >>> Basically it was suggested to automarically mask NaN (and Inf ?) when >>> creating ma. >>> I'm sure you already thought of this on this list and was curious to >>> know why you decided not to do it. >>> >>> >> >> Because it's always best to let the user decide what to do with his/her >> data and not impose anything ? >> >> Masking a point doesn't necessarily mean that the point is invalid (in the >> sense of NaNs/Infs), just that it doesn't satisfy some particular condition. >> In that sense, masks act as selecting tools. >> >> By forcing invalid data to be masked at the creation of an array, you run >> the risk to tamper with the (potential) physical meaning of the mask you >> have given as input, and/or miss the fact that some data are actually >> invalid when you don't expect it to be. >> >> Let's take an example: I want to analyze sea surface temperatures at the >> world scale. The data comes as a regular 2D ndarray, with NaNs for missing >> or invalid data. In a first step, I create a masked array of this data, >> filtering out the land masses by a predefined geographical mask. The >> remaining NaNs in the masked array indicate areas where the sensor failed... >> It's an important information I would probably have missed by masking all >> the NaNs at first... >> >> >> As Eric F. suggested, you can use numpy.ma.masked_invalid to create a >> masked array with NaNs/Infs filtered out: >> >> >> >>> import numpy as np,. numpy.ma as ma >>>>> x = np.array([1,2,None,4], dtype=float) >>>>> x >>>>> >>>>> >>>> array([ 1., 2., NaN, 4.]) >> >> >>> mx = ma.masked_invalid(x) >>>>> mx >>>>> >>>>> >>>> masked_array(data = [1.0 2.0 -- 4.0], >> mask = [False False True False], >> fill_value=1e+20) >> >> Note that the underlying data still has NaNs/Infs: >> >> >>> mx._data >>>>> >>>>> >>>> array([ 1., 2., NaN, 4.]) >> >> You can also use the ma.fix_invalid function: it creates a mask where the >> data is not finite (NaNs/Infs), and set the corresponding points to >> fill_value. >> >> >>> mx = ma.fix_invalid(x, fill_value=999) >>>>> mx >>>>> >>>>> >>>> masked_array(data = [1.0 2.0 -- 4.0], >> mask = [False False True False], >> fill_value=1e+20) >> >> >>> mx._data >>>>> >>>>> >>>> array([ 1., 2., 999., 4.]) >> >> >> The advantage of the second approach is that you no longer have NaNs/Infs >> in the underlying data, which speeds things up during computation. The >> obvious disadvantage is that you no longer know where the data was >> invalid... >> >> >> > > > > > ---------- Forwarded message ---------- > From: "Christopher Burns" <[EMAIL PROTECTED]> > To: "Discussion of Numerical Python" <[email protected]> > Date: Fri, 25 Jul 2008 11:48:13 -0700 > Subject: [Numpy-discussion] numpy-1.1.1rc2 Mac binary - Please Test. > Reminder, please test the Mac installer for rc2 so we have time to fix any > bugs before the release next week. > > Also, I committed my build script to the trunk/tools/osxbuild. bdist_mpkg > 0.4.3 is required. > > Thank you, > Chris > > On Thu, Jul 24, 2008 at 11:03 AM, Jarrod Millman <[EMAIL PROTECTED]> > wrote: > >> Hello, >> >> The 1.1.1rc2 is now available: >> http://svn.scipy.org/svn/numpy/tags/1.1.1rc2 >> >> The source tarball is here: >> http://cirl.berkeley.edu/numpy/numpy-1.1.1rc2.tar.gz >> >> Here is the universal Mac binary: >> http://cirl.berkeley.edu/numpy/numpy-1.1.1rc2-py2.5-macosx10.5.dmg >> >> David Cournapeau will be creating a 1.1.1rc2 Windows binary in next few >> days. >> >> Please test this release ASAP and let us know if there are any >> problems. If there are no show stoppers, this will likely become the >> 1.1.1 release. >> >> Thanks, >> >> -- >> Jarrod Millman >> Computational Infrastructure for Research Labs >> 10 Giannini Hall, UC Berkeley >> phone: 510.643.4014 >> http://cirl.berkeley.edu/ >> _______________________________________________ >> Numpy-discussion mailing list >> [email protected] >> http://projects.scipy.org/mailman/listinfo/numpy-discussion >> > > > > ---------- Forwarded message ---------- > From: "Alan McIntyre" <[EMAIL PROTECTED]> > To: "Discussion of Numerical Python" <[email protected]> > Date: Fri, 25 Jul 2008 15:00:42 -0400 > Subject: Re: [Numpy-discussion] numpy-1.1.1rc2 Mac binary - Please Test. > On Fri, Jul 25, 2008 at 2:48 PM, Christopher Burns <[EMAIL PROTECTED]> > wrote: > > Reminder, please test the Mac installer for rc2 so we have time to fix > any > > bugs before the release next week. > > I just tried it; it installs with no problems and tests run with no > failures. > > > > ---------- Forwarded message ---------- > From: "Frank Lagor" <[EMAIL PROTECTED]> > To: [email protected] > Date: Fri, 25 Jul 2008 15:32:56 -0400 > Subject: [Numpy-discussion] curious problem with SVD > Perhaps I do not understand something properly, if so could someone please > explain the behavior I notice with numpy.linalg.svd when acting on arrays. > It gives the incorrect answer, but works fine with matrices. My numpy is > 1.1.0. > > >>> R = n.array([[3.6,.35],[.35,1.8]]) > >>> V,D,W = n.linalg.svd(R) > >>> V*n.diag(D)*W.transpose() > array([[ 3.5410365 , 0. ], > [ 0. , 1.67537611]]) > >>> R = n.matrix([[3.6,.35],[.35,1.8]]) > >>> V,D,W = n.linalg.svd(R) > >>> V*n.diag(D)*W.transpose() > matrix([[ 3.6 , 0.35], > [ 0.35, 1.8 ]]) > > Thanks in advance, > Frank > -- > Frank Lagor > Ph.D. Candidate > Mechanical Engineering and Applied Mechanics > University of Pennsylvania > > > ---------- Forwarded message ---------- > From: "Keith Goodman" <[EMAIL PROTECTED]> > To: "Discussion of Numerical Python" <[email protected]> > Date: Fri, 25 Jul 2008 12:36:28 -0700 > Subject: Re: [Numpy-discussion] curious problem with SVD > On Fri, Jul 25, 2008 at 12:32 PM, Frank Lagor <[EMAIL PROTECTED]> > wrote: > > Perhaps I do not understand something properly, if so could someone > please > > explain the behavior I notice with numpy.linalg.svd when acting on > arrays. > > It gives the incorrect answer, but works fine with matrices. My numpy is > > 1.1.0. > > > >>>> R = n.array([[3.6,.35],[.35,1.8]]) > >>>> V,D,W = n.linalg.svd(R) > >>>> V*n.diag(D)*W.transpose() > > array([[ 3.5410365 , 0. ], > > [ 0. , 1.67537611]]) > >>>> R = n.matrix([[3.6,.35],[.35,1.8]]) > >>>> V,D,W = n.linalg.svd(R) > >>>> V*n.diag(D)*W.transpose() > > matrix([[ 3.6 , 0.35], > > [ 0.35, 1.8 ]]) > > '*' does element-by-element multiplication for arrays but matrix > multiplication for matrices. > > > > ---------- Forwarded message ---------- > From: "Keith Goodman" <[EMAIL PROTECTED]> > To: "Discussion of Numerical Python" <[email protected]> > Date: Fri, 25 Jul 2008 12:39:23 -0700 > Subject: Re: [Numpy-discussion] curious problem with SVD > On Fri, Jul 25, 2008 at 12:36 PM, Keith Goodman <[EMAIL PROTECTED]> > wrote: > > On Fri, Jul 25, 2008 at 12:32 PM, Frank Lagor <[EMAIL PROTECTED]> > wrote: > >> Perhaps I do not understand something properly, if so could someone > please > >> explain the behavior I notice with numpy.linalg.svd when acting on > arrays. > >> It gives the incorrect answer, but works fine with matrices. My numpy > is > >> 1.1.0. > >> > >>>>> R = n.array([[3.6,.35],[.35,1.8]]) > >>>>> V,D,W = n.linalg.svd(R) > >>>>> V*n.diag(D)*W.transpose() > >> array([[ 3.5410365 , 0. ], > >> [ 0. , 1.67537611]]) > >>>>> R = n.matrix([[3.6,.35],[.35,1.8]]) > >>>>> V,D,W = n.linalg.svd(R) > >>>>> V*n.diag(D)*W.transpose() > >> matrix([[ 3.6 , 0.35], > >> [ 0.35, 1.8 ]]) > > > > '*' does element-by-element multiplication for arrays but matrix > > multiplication for matrices. > > As a check (for the array case): > > >> n.dot(V, n.dot(n.diag(D), W.transpose())) # That's hard to read! > > array([[ 3.6 , 0.35], > [ 0.35, 1.8 ]]) > > > > ---------- Forwarded message ---------- > From: "Robert Kern" <[EMAIL PROTECTED]> > To: "Discussion of Numerical Python" <[email protected]> > Date: Fri, 25 Jul 2008 14:50:24 -0500 > Subject: Re: [Numpy-discussion] curious problem with SVD > On Fri, Jul 25, 2008 at 14:32, Frank Lagor <[EMAIL PROTECTED]> wrote: > > Perhaps I do not understand something properly, if so could someone > please > > explain the behavior I notice with numpy.linalg.svd when acting on > arrays. > > It gives the incorrect answer, but works fine with matrices. My numpy is > > 1.1.0. > > > >>>> R = n.array([[3.6,.35],[.35,1.8]]) > >>>> V,D,W = n.linalg.svd(R) > >>>> V*n.diag(D)*W.transpose() > > For regular arrays, * is element-wise multiplication, not matrix > multiplication. For matrix objects, * is matrix multiplication. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma that is made terrible by our own mad attempt to interpret it as > though it had an underlying truth." > -- Umberto Eco > > > > ---------- Forwarded message ---------- > From: "Arnar Flatberg" <[EMAIL PROTECTED]> > To: "Discussion of Numerical Python" <[email protected]> > Date: Fri, 25 Jul 2008 21:53:09 +0200 > Subject: Re: [Numpy-discussion] curious problem with SVD > > On Fri, Jul 25, 2008 at 9:39 PM, Keith Goodman <[EMAIL PROTECTED]> > wrote: > >> On Fri, Jul 25, 2008 at 12:36 PM, Keith Goodman <[EMAIL PROTECTED]> >> wrote: >> > On Fri, Jul 25, 2008 at 12:32 PM, Frank Lagor <[EMAIL PROTECTED]> >> wrote: >> >> Perhaps I do not understand something properly, if so could someone >> please >> >> explain the behavior I notice with numpy.linalg.svd when acting on >> arrays. >> >> It gives the incorrect answer, but works fine with matrices. My numpy >> is >> >> > '*' does element-by-element multiplication for arrays but matrix >> > multiplication for mat >> >> n.dot(V, n.dot(n.diag(D), W.transpose())) # That's hard to read! > > > Just two small points: > > 1.) Broadcasting may be easier on the eye ... well, atleast when you have > gotten used to it > Then the above is np.dot(V*D, W) > > 2.) Also, note that the right hand side eigenvectors in numpy's svd routine > is ordered by rows! > Yes, I know this is confusing as it is different from just about any other > linear algebra software out there, but the documentation is clear. It is > also a little inconsistent with eig and eigh, some more experienced user can > probably answer on why it is like that? > > Arnar > > > >> <http://projects.scipy.org/mailman/listinfo/numpy-discussion> >> > > _______________________________________________ > Numpy-discussion mailing list > [email protected] > http://projects.scipy.org/mailman/listinfo/numpy-discussion > > -- Frank Lagor Ph.D. Candidate Mechanical Engineering and Applied Mechanics University of Pennsylvania
_______________________________________________ Numpy-discussion mailing list [email protected] http://projects.scipy.org/mailman/listinfo/numpy-discussion
