[Numpy-discussion] More on Summer NumPy Doc Marathon
Hi again, folks. I have a special request. Part of the vision for my job is that I'll focus my writing efforts on the docs no one else is gung-ho to work on. So, even if you're not quite ready to commit, if you're leaning toward volunteering to be a team lead for one (or more) categories, please let me know which one(s) (off list, if you prefer) so I can get an initial idea of what the leftovers are going to be. Thanks! DG ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Definitions of pv, fv, nper, pmt, and rate
--- On Mon, 6/8/09, Robert Kern robert.k...@gmail.com wrote: The OpenFormula standard is probably better: http://www.oasis-open.org/committees/documents.php?wg_abbrev=office-formula -- Robert Kern OK, thanks Robert (as always); I'll go ahead and use this until/unless someone provide a printed reference. Thanks again. DG ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance matrix multiplication vs. matlab
On Mon, Jun 8, 2009 at 7:14 PM, David Warde-Farleyd...@cs.toronto.edu wrote: On 8-Jun-09, at 8:33 AM, Jason Rennie wrote: Note that EM can be very slow to converge: That's absolutely true, but EM for PCA can be a life saver in cases where diagonalizing (or even computing) the full covariance matrix is not a realistic option. Diagonalization can be a lot of wasted effort if all you care about are a few leading eigenvectors. EM also lets you deal with missing values in a principled way, which I don't think you can do with standard SVD. EM certainly isn't a magic bullet but there are circumstances where it's appropriate. I'm a big fan of the ECG paper too. :) Hi, I've been following this with interest... although I'm not really familiar with the area. At the risk of drifting further off topic I wondered if anyone could recommend an accessible review of these kind of dimensionality reduction techniques... I am familiar with PCA and know of diffusion maps and ICA and others, but I'd never heard of EM and I don't really have any idea how they relate to each other and which might be better for one job or the other... so some sort of primer would be really handy. Cheers Robin ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance matrix multiplication vs. matlab
2009/6/9 Robin robi...@gmail.com: On Mon, Jun 8, 2009 at 7:14 PM, David Warde-Farleyd...@cs.toronto.edu wrote: On 8-Jun-09, at 8:33 AM, Jason Rennie wrote: Note that EM can be very slow to converge: That's absolutely true, but EM for PCA can be a life saver in cases where diagonalizing (or even computing) the full covariance matrix is not a realistic option. Diagonalization can be a lot of wasted effort if all you care about are a few leading eigenvectors. EM also lets you deal with missing values in a principled way, which I don't think you can do with standard SVD. EM certainly isn't a magic bullet but there are circumstances where it's appropriate. I'm a big fan of the ECG paper too. :) Hi, I've been following this with interest... although I'm not really familiar with the area. At the risk of drifting further off topic I wondered if anyone could recommend an accessible review of these kind of dimensionality reduction techniques... I am familiar with PCA and know of diffusion maps and ICA and others, but I'd never heard of EM and I don't really have any idea how they relate to each other and which might be better for one job or the other... so some sort of primer would be really handy. Hi, Check Ch. Bishop publication on Probabilistic Principal Components Analysis, you have there the parallel between the two (EM is in fact just a way of computing PPCA, and with some Gaussian assumptions, you get PCA). Matthieu -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance matrix multiplication vs. matlab
Robin wrote: On Mon, Jun 8, 2009 at 7:14 PM, David Warde-Farleyd...@cs.toronto.edu wrote: On 8-Jun-09, at 8:33 AM, Jason Rennie wrote: Note that EM can be very slow to converge: That's absolutely true, but EM for PCA can be a life saver in cases where diagonalizing (or even computing) the full covariance matrix is not a realistic option. Diagonalization can be a lot of wasted effort if all you care about are a few leading eigenvectors. EM also lets you deal with missing values in a principled way, which I don't think you can do with standard SVD. EM certainly isn't a magic bullet but there are circumstances where it's appropriate. I'm a big fan of the ECG paper too. :) Hi, I've been following this with interest... although I'm not really familiar with the area. At the risk of drifting further off topic I wondered if anyone could recommend an accessible review of these kind of dimensionality reduction techniques... I am familiar with PCA and know of diffusion maps and ICA and others, but I'd never heard of EM and I don't really have any idea how they relate to each other and which might be better for one job or the other... so some sort of primer would be really handy. I think the biggest problem is the 'babel tower' aspect of machine learning (the expression is from David H. Wolpert I believe), and practitioners in different subfields often use totally different words for more or less the same concepts (and many keep being rediscovered). For example, what ML people call PCA is called Karhunen Loéve in signal processing, and the concepts are quite similar. Anyway, the book from Bishop is a pretty good reference by one of the leading researcher: http://research.microsoft.com/en-us/um/people/cmbishop/prml/ It can be read without much background besides basic 1st year calculus/linear algebra. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance matrix multiplication vs. matlab
David Cournapeau wrote: I think the biggest problem is the 'babel tower' aspect of machine learning (the expression is from David H. Wolpert I believe), and practitioners in different subfields often use totally different words for more or less the same concepts (and many keep being rediscovered). For example, what ML people call PCA is called Karhunen Loéve in signal processing, and the concepts are quite similar. Anyway, the book from Bishop is a pretty good reference by one of the leading researcher: http://research.microsoft.com/en-us/um/people/cmbishop/prml/ Should have mentioned that it is the same Bishop as mentioned by Matthieu, and that chapter 12 deals with latent models with continuous latent variable, which is one way to consider PCA in a probabilistic framework. David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] setmember1d_nu
Robert Cimrman cimrman3 at ntc.zcu.cz writes: I'd really like to see the setmember1d_nu function in ticket 1036 get into numpy. There's a patch waiting for review that including tests: http://projects.scipy.org/numpy/ticket/1036 Is there anything I can do to help get it applied? I guess I could commit it, if you review the patch and it works for you. Obviously, I cannot review it myself, but my SVN access may still work :) Thanks for the review, it is in! r. Great - thanks! People often post to the list asking for this functionality, so it's nice to get it into numpy (whatever it ends up being called). Neil ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance matrix multiplication vs. matlab
2009/6/9 David Cournapeau da...@ar.media.kyoto-u.ac.jp: Anyway, the book from Bishop is a pretty good reference by one of the leading researcher: http://research.microsoft.com/en-us/um/people/cmbishop/prml/ It can be read without much background besides basic 1st year calculus/linear algebra. Bishop's book could be confusing at times, so I would also recommend going back to the original papers. It is sometimes easier to learn *with* researchers than from them! Cheers Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] error with large memmap
I'm getting the error OverflowError: cannot fit 'long' into an index-sized integer when I try to memmap a 6gb file top of the stack trace is mm = mmap.mmap(fid.fileno(), bytes, access=acc) where bytes = 652800L I thought that 64-bit systems with python2.5 could memmap large files. I'm running the latest EPD python distribution (4.2.30201), which uses python 2.5.4 and numpy 1.2.1 Macbook Pro Core 2 Duo, OS X 10.5.6 ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] setmember1d_nu
Neil Crighton wrote: Robert Cimrman cimrman3 at ntc.zcu.cz writes: I'd really like to see the setmember1d_nu function in ticket 1036 get into numpy. There's a patch waiting for review that including tests: http://projects.scipy.org/numpy/ticket/1036 Is there anything I can do to help get it applied? I guess I could commit it, if you review the patch and it works for you. Obviously, I cannot review it myself, but my SVN access may still work :) Thanks for the review, it is in! r. Great - thanks! People often post to the list asking for this functionality, so it's nice to get it into numpy (whatever it ends up being called). Thank you for starting the discussion :) ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] improving arraysetops
Hi, I am starting a new thread, so that it reaches the interested people. Let us discuss improvements to arraysetops (array set operations) at [1] (allowing non-unique arrays as function arguments, better naming conventions and documentation). r. [1] http://projects.scipy.org/numpy/ticket/1133 ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to remove fortran-like loops with numpy?
2009/6/9 Stéfan van der Walt ste...@sun.ac.za: Hi Juan ---cut --- As you can see, it is very simple, but it takes several seconds running just to create a 200x200 plot. Fortran takes same time to create a 2000x2000 plot, around 100 times faster... So the question is, do you know how to programme this in a Python-like fashion in order to improve seriously the performance? Here is another version, similar to Robert's, that I wrote up for the documentation project last year: http://mentat.za.net/numpy/intro/intro.html We never used it, but I still like the pretty pictures :-) +1 Hi Stéfan, this does look really nice !! Could it be put somewhere more prominent !? How about onto the SciPy site with a newcomers start here-link right on the first page My 2 cents, Sebastian Haase ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] error with large memmap
On Tue, Jun 9, 2009 at 2:32 AM, John Schulman jos...@caltech.edu wrote: I'm getting the error OverflowError: cannot fit 'long' into an index-sized integer when I try to memmap a 6gb file top of the stack trace is mm = mmap.mmap(fid.fileno(), bytes, access=acc) where bytes = 652800L I thought that 64-bit systems with python2.5 could memmap large files. I'm running the latest EPD python distribution (4.2.30201), which uses python 2.5.4 and numpy 1.2.1 Macbook Pro Core 2 Duo, OS X 10.5.6 Is your python 64 bits? Try: file `which python` and see what it says. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] error with large memmap
Charles R Harris wrote: On Tue, Jun 9, 2009 at 2:32 AM, John Schulman jos...@caltech.edu mailto:jos...@caltech.edu wrote: I'm getting the error OverflowError: cannot fit 'long' into an index-sized integer when I try to memmap a 6gb file top of the stack trace is mm = mmap.mmap(fid.fileno(), bytes, access=acc) where bytes = 652800L I thought that 64-bit systems with python2.5 could memmap large files. I'm running the latest EPD python distribution (4.2.30201), which uses python 2.5.4 and numpy 1.2.1 Macbook Pro Core 2 Duo, OS X 10.5.6 Is your python 64 bits? Try: file `which python` This is even better: python -c import platform; print platform.machine() as mac os x can be confusing with fat binaries and all :) David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Definitions of pv, fv, nper, pmt, and rate
On Tue, Jun 9, 2009 at 1:14 AM, josef.p...@gmail.com wrote: On Tue, Jun 9, 2009 at 12:51 AM, d_l_goldsm...@yahoo.com wrote: --- On Mon, 6/8/09, Skipper Seabold jsseab...@gmail.com wrote: I forgot the last payment (which doesn't earn any interest), so one more 100. So in fact they're not in agreement? pretty soon. I don't have a more permanent reference for fv offhand, but it should be in any corporate finance text etc. Most of these type of formulas use basic results of geometric series to simplify. Let me be more specific about the difference between what we have and what I'm finding in print. Essentially, it boils down to this: in every source I've found, two different present/future values are discussed, that for a single amount, and that for a constant (i.e., not even the first payment is allowed to be different) periodic payment. I have not been able to find a single printed reference that gives a formula for (or even discusses, for that matter) the combination of these two, which is clearly what we have implemented (and which is, just as clearly, actually seen in practice). These are the two most basic building blocks of time value problems, discounting one cash flow and an annuity. There are *plenty* of examples and use cases for uneven cash flows or for providing a given pv or fv. Without even getting into actual financial contracts, suppose I have an investment account that already has $10,000 and I plan to add $500 every month and earn 4%. Then we would need something like fv to tell me how much this will be worth after 180 months. I don't necessarily need a reference to tell me this would be useful to know. Now, my lazy side simply hopes that my stridency will finally cause someone to pipe up and say look, dummy, it's in Schmoe, Joe, 2005. Advanced Financial Practice. Financial Press, NY NY. There's your reference; find it and look it up if you don't trust me and then I'll feel like we've at least covered our communal rear-end. But my more conscientious side worries that, if I've had so much trouble finding our more advanced definition (and I have tried, believe me), then I'm concerned that what your typical student (for example) is most likely to encounter is one of those simpler definitions, and thus get confused (at best) if they look at our help doc and find quite a different (at least superficially) definition (or worse, don't look at the help doc, and either can't get the function to work because the required number of inputs doesn't match what they're expecting from their text, or somehow manage to get it to work, but get an answer very different from that given in other sources, e.g., the answers in the back of their text.) I don't know that these are formulas per se, rather than convenience functions for typical use cases. That's why they're in spreadsheets in the first place. They also follow the behavior of financial calculators, where you typically have to input a N, I/Y, PMT, PV and FV (even if one of these last two values is zero). If you need a textbook reference, as I said before you could literally pick up any corporate finance text and derive these functions from the basics. Try having a look at some end of chapter questions (or financial calculator handbook) to get an idea of when and how they'd actually be used. One obvious answer to this dilemma is to explain this discrepancy in the help doc, but then we have to explain - clearly and lucidly, mind you - how one uses our functions for the two simpler cases, how/why the formula we use is the combination of the other two, etc. (it's rather hard to anticipate, for me at least, all the possible confusions this discrepancy might create) and in any event, somehow I don't really think something so necessarily elaborate is appropriate in this case. So, again, given that fv and pv (and by extension, nper, pmt, and rate) have multiple definitions floating around out there, I sincerely think we should punt (my apologies to those unfamiliar w/ the American football metaphor), i.e., rid ourselves of this nightmare, esp. in light of what I feel are compelling, independent arguments against the inclusion of these functions in this library in the first place. Sorry for my stridency, and thank you for your time and patience. I don't think that there are multiple definitions of these (very simple) functions floating around, but rather different assumptions/implementations that lead to ever so slightly different results. My plan for the additions and when checking the existing ones is to derive the result, so that we know what's going on. Once you state your assumptions, the result will be clearly one way or another. This would be my way of covering our functions. I derived the result, so here's what's going on, here's a use case to have a look at as an example. Then we should be fine. It's not that I don't appreciate your concern
Re: [Numpy-discussion] error with large memmap
OK looks like that was the issue $ python -c import platform; print platform.machine() i386 Thanks On Tue, Jun 9, 2009 at 2:52 AM, David Cournapeauda...@ar.media.kyoto-u.ac.jp wrote: Charles R Harris wrote: On Tue, Jun 9, 2009 at 2:32 AM, John Schulman jos...@caltech.edu mailto:jos...@caltech.edu wrote: I'm getting the error OverflowError: cannot fit 'long' into an index-sized integer when I try to memmap a 6gb file top of the stack trace is mm = mmap.mmap(fid.fileno(), bytes, access=acc) where bytes = 652800L I thought that 64-bit systems with python2.5 could memmap large files. I'm running the latest EPD python distribution (4.2.30201), which uses python 2.5.4 and numpy 1.2.1 Macbook Pro Core 2 Duo, OS X 10.5.6 Is your python 64 bits? Try: file `which python` This is even better: python -c import platform; print platform.machine() as mac os x can be confusing with fat binaries and all :) David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Adding zero-dimension arrays, bug?
Hi, Numpy let's me define arrays with zero rows and/or columns, and that's wanted behaviour from what I have read in discussions. However, I can add an array with zero rows to an array with one row (but not more), resulting in another zero row array, like so: In: a = zeros((4,0)) In: a Out: array([], shape=(4, 0), dtype=float64) In: b = zeros((4,1)) In: b Out: array([[ 0.], [ 0.], [ 0.], [ 0.]]) In: a + b Out: array([], shape=(4, 0), dtype=float64) Is this a bug? This should give a shape mismatch error, shouldn't? Cheers David -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] error with large memmap
What's the best way to install a 64-bit python alongside my existing installation? On Tue, Jun 9, 2009 at 7:32 AM, John Schulmanjos...@caltech.edu wrote: OK looks like that was the issue $ python -c import platform; print platform.machine() i386 Thanks On Tue, Jun 9, 2009 at 2:52 AM, David Cournapeauda...@ar.media.kyoto-u.ac.jp wrote: Charles R Harris wrote: On Tue, Jun 9, 2009 at 2:32 AM, John Schulman jos...@caltech.edu mailto:jos...@caltech.edu wrote: I'm getting the error OverflowError: cannot fit 'long' into an index-sized integer when I try to memmap a 6gb file top of the stack trace is mm = mmap.mmap(fid.fileno(), bytes, access=acc) where bytes = 652800L I thought that 64-bit systems with python2.5 could memmap large files. I'm running the latest EPD python distribution (4.2.30201), which uses python 2.5.4 and numpy 1.2.1 Macbook Pro Core 2 Duo, OS X 10.5.6 Is your python 64 bits? Try: file `which python` This is even better: python -c import platform; print platform.machine() as mac os x can be confusing with fat binaries and all :) David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] error with large memmap
John Schulman wrote: What's the best way to install a 64-bit python alongside my existing installation? It is a bit complicated because you need to build your own python interpreter (the python.org one does not handle 64 bits AFAIK). You could just install your python somewhere in your $HOME for example, and whenever you use this python interpreter to build a package, it will install it inside the site-packages of this python - so no clash with existing python interpreters. There are also things like virtualenv and co which can be considered if you need more than a couple of packages, cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] second 2d fft gives the same result as fft+ifft
I tried to calculate the second fourier transformation of an image with the following code below: --- import pylab import numpy ### Create a simple image fx = numpy.zeros( 128**2 ).reshape(128,128).astype( numpy.float ) for i in xrange(8): for j in xrange(8): fx[i*8+16][j*8+16] = 1.0 ### Fourier Transformations Ffx = numpy.copy( numpy.fft.fft2( fx ).real ) # 1st fourier FFfx = numpy.copy( numpy.fft.fft2( Ffx ).real ) # 2nd fourier IFfx = numpy.copy( numpy.fft.ifft2( Ffx ).real ) # inverse fourier ### Display result pylab.figure( 1, figsize=(8,8), dpi=125 ) pylab.subplot(221) pylab.imshow( fx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( fx ) pylab.subplot(222) pylab.imshow( Ffx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( Ffx ) pylab.subplot(223) pylab.imshow( FFfx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( FFfx ) pylab.subplot(224) pylab.imshow( IFfx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( IFfx ) pylab.show() --- On my computer FFfx is the same as IFfx. but why? I uploaded a screenshot about my result here: http://server6.theimagehosting.com/image.php?img=second_fourier.png Bela -- View this message in context: http://www.nabble.com/second-2d-fft-gives-the-same-result-as-fft%2Bifft-tp23945026p23945026.html Sent from the Numpy-discussion mailing list archive at Nabble.com. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] second 2d fft gives the same result as fft+ifft
Hi, Is it really ? You only show the imaginary part of the FFT, so you can't be sure of what you are saying. Don't forget that the only difference between FFT and iFFT is (besides of teh scaling factor) a minus sign in the exponent. Matthieu 2009/6/9 bela bela.miha...@gmail.com: I tried to calculate the second fourier transformation of an image with the following code below: --- import pylab import numpy ### Create a simple image fx = numpy.zeros( 128**2 ).reshape(128,128).astype( numpy.float ) for i in xrange(8): for j in xrange(8): fx[i*8+16][j*8+16] = 1.0 ### Fourier Transformations Ffx = numpy.copy( numpy.fft.fft2( fx ).real ) # 1st fourier FFfx = numpy.copy( numpy.fft.fft2( Ffx ).real ) # 2nd fourier IFfx = numpy.copy( numpy.fft.ifft2( Ffx ).real ) # inverse fourier ### Display result pylab.figure( 1, figsize=(8,8), dpi=125 ) pylab.subplot(221) pylab.imshow( fx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( fx ) pylab.subplot(222) pylab.imshow( Ffx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( Ffx ) pylab.subplot(223) pylab.imshow( FFfx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( FFfx ) pylab.subplot(224) pylab.imshow( IFfx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( IFfx ) pylab.show() --- On my computer FFfx is the same as IFfx. but why? I uploaded a screenshot about my result here: http://server6.theimagehosting.com/image.php?img=second_fourier.png Bela -- View this message in context: http://www.nabble.com/second-2d-fft-gives-the-same-result-as-fft%2Bifft-tp23945026p23945026.html Sent from the Numpy-discussion mailing list archive at Nabble.com. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] error with large memmap
On Tue, Jun 9, 2009 at 8:51 AM, John Schulman jos...@caltech.edu wrote: What's the best way to install a 64-bit python alongside my existing installation? There was a long thread about that a while back. Try a search on the archives. It wasn't all that easy, though. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-dev] More on Summer NumPy Doc Marathon
Thanks, Stefan. The lists you suggest already exist (more or less, depending on the thing, i.e., list of categories, completely, prioritized list of individual items, sort of, at least w/in the categories) on the Milestones page (that's essentially what the Milestones page is) and the list of individual items is far too long to duplicate here, but for everyone's convenience I'll provide the list of categories (at least those for which the goal has not been, or is not close to being, met, which is most of them): Data type investigation Fourier transforms Linear algebra Error handling Financial functions Functional operations Help routines Indexing Input/Output Logic, comparisons etc. Polynomials Random number generation Other random operations Boolean set operations Searching Sorting Statistics Comparison Window functions Sums, interpolation, gradients, etc Arithmetic + basic functions I Arithmetic + basic functions II Arithmetic + basic functions III Masked arrays Masked arrays, II Masked arrays, III Masked arrays, IV Operations on masks Even more MA functions I Even more MA functions II Numpy internals C-types Other math The matrix library Numarray compatibility Numeric compatibility Other array subclasses Matrix subclass Ndarray Ndarray, II Dtypes Ufunc Scalar base class Scalar types Comments: 0) The number of individual items in each of these categories varies from one to a few dozen or so 1) Omitted are a few meta-categories, e.g., Routines, Basic Objects, etc. 2) IMO, there are still too many of these (at least too many to not be intimidating in the manner Stefan has implied); I had it in mind to try to create an intermediate level of organization, i.e., meso-categories, but I couldn't really justify it on grounds other than there are simply still too many categories to be unintimidating, so I was advised against usage of time in that endeavor. However, if there's an outpouring of support for me doing that, it would fall on sympathetic ears. As far as prioritizing individual items, my opinion is that team leads should do that (or not, as they deem appropriate) - I wouldn't presume to know enough to do that in most cases. However, if people want to furnish me with suggested prioritizations, I'd be happy to be the one to edit the Wiki to reflect these. DG --- On Tue, 6/9/09, Stéfan van der Walt ste...@sun.ac.za wrote: From: Stéfan van der Walt ste...@sun.ac.za Subject: Re: [SciPy-dev] More on Summer NumPy Doc Marathon To: SciPy Developers List scipy-...@scipy.org Date: Tuesday, June 9, 2009, 1:34 AM Hi David 2009/6/9 David Goldsmith d_l_goldsm...@yahoo.com: Hi again, folks. I have a special request. Part of the vision for my job is that I'll focus my writing efforts on the docs no one else is gung-ho to work on. So, even if you're not quite ready to commit, if you're leaning toward volunteering to be a team lead for one (or more) categories, please let me know which one(s) (off list, if you prefer) so I can get an initial idea of what the leftovers are going to be. Thanks! That's a pretty wide question. Maybe you could post a list of categories and ask who would be willing to mentor and write on each? For the writing, we could decide on a prioritised list of functions, publish that list and then document those entries one by one (i.e. make the tasks small enough so that people don't run away screaming). Cheers Stéfan ___ Scipy-dev mailing list scipy-...@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Definitions of pv, fv, nper, pmt, and rate
--- On Tue, 6/9/09, Skipper Seabold jsseab...@gmail.com wrote: These are the two most basic building blocks of time value problems, discounting one cash flow and an annuity. There are *plenty* of examples and use cases for uneven cash flows or for providing a given pv or fv. Without even getting into actual financial contracts, suppose I have an investment account that already has $10,000 and I plan to add $500 every month and earn 4%. Then we would need something like fv to tell me how much this will be worth after 180 months. I don't necessarily need a reference to tell me this would be useful to know. Use case examples aren't the problem; worked out examples combining these two principles aren't the problem; usefulness isn't the problem; the problem is one of meeting a particular reference standard. I don't know that these are formulas per se, rather than Except that we do provide a formula in our help doc; perhaps the solution is to get rid of that and include an explanation of how our function combines the two basic formulae (for which I do have a hard-copy reference: Gitman, L. J., 2003. Principals of Managerial Finance (Brief), 3rd Ed. Pearson Education) to handle the more general case. FV (even if one of these last two values is zero). If you need a textbook reference, as I said before you could literally pick up any corporate finance text and derive these functions from the basics. I don't question that, what I question is the appropriateness of such derivation in numpy's help doc; as I see it, one of the purposes of providing references in our situation is precisely to avoid having to include derivations in our help doc. But it's all moot, as Robert has furnished me with a reference specifically for the formula we do have, and, although it's an electronic reference, it seems stable enough to warrant an exception to the references should be hard-copy AMAP policy. DG ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-dev] More on Summer NumPy Doc Marathon
Thanks, Bruce. --- On Tue, 6/9/09, Bruce Southey bsout...@gmail.com wrote: Hi, Great. Can you provide the actual colors at the start for : * Edit white, light gray, or yellow * Don't edit dark gray While not color-blind, not all browsers render the same colors on all operating systems etc. What are you using for light gray or is that meant to be blue. If it is 'blue' then what does it mean? It appears to be the same color used on the Front Page to say 'Proofed'. Yes, by all means, I agree 100% (I'm having the same problem). :-) In fact, I think (and have thought) that the light and dark grey and cyan are all too close to each other - anyone object to me replacing the greys w/ orange and lavender? What does the 'green' color mean? The links says 'Reviewed (needs proof)' but how does one say 'proofed'. Cyan (if I understand you correctly). Also the milestone link from the Front Page does not go anywhere: http://docs.scipy.org/numpy/Front%20Page/#milestones Ooops, accidentally broke that editing the page to make the Milestones link more prominent - I'll fix it imminently. Thanks again, DG Bruce ___ Scipy-dev mailing list scipy-...@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-dev] More on Summer NumPy Doc Marathon
Also the milestone link from the Front Page does not go anywhere: http://docs.scipy.org/numpy/Front%20Page/#milestones Fixed. DG ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance matrix multiplication vs. matlab
On 9-Jun-09, at 3:54 AM, David Cournapeau wrote: For example, what ML people call PCA is called Karhunen Loéve in signal processing, and the concepts are quite similar. Yup. This seems to be a nice set of review notes: http://www.ece.rutgers.edu/~orfanidi/ece525/svd.pdf And going further than just PCA/KLT, tying it together with maximum likelihood factor analysis/linear dynamical systems/hidden Markov models, http://www.cs.toronto.edu/~roweis/papers/NC110201.pdf David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] second 2d fft gives the same result as fft+ifft
--- On Tue, 6/9/09, Matthieu Brucher matthieu.bruc...@gmail.com wrote: Hi, Is it really ? You only show the imaginary part of the FFT, so you can't be sure of what you are saying. Indeed, is there not a label for a function f which satisfies Im(iFFT(f)) = Im(FFT^2(f)), Re(iFFT(f)) != Re(FFT^2(f))? (And similarly if Im and Re roles are reversed.) Seems like the class of such functions (if any exist) might have some interesting properties... DG Don't forget that the only difference between FFT and iFFT is (besides of teh scaling factor) a minus sign in the exponent. Matthieu 2009/6/9 bela bela.miha...@gmail.com: I tried to calculate the second fourier transformation of an image with the following code below: --- import pylab import numpy ### Create a simple image fx = numpy.zeros( 128**2 ).reshape(128,128).astype( numpy.float ) for i in xrange(8): for j in xrange(8): fx[i*8+16][j*8+16] = 1.0 ### Fourier Transformations Ffx = numpy.copy( numpy.fft.fft2( fx ).real ) # 1st fourier FFfx = numpy.copy( numpy.fft.fft2( Ffx ).real ) # 2nd fourier IFfx = numpy.copy( numpy.fft.ifft2( Ffx ).real ) # inverse fourier ### Display result pylab.figure( 1, figsize=(8,8), dpi=125 ) pylab.subplot(221) pylab.imshow( fx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( fx ) pylab.subplot(222) pylab.imshow( Ffx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( Ffx ) pylab.subplot(223) pylab.imshow( FFfx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( FFfx ) pylab.subplot(224) pylab.imshow( IFfx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( IFfx ) pylab.show() --- On my computer FFfx is the same as IFfx. but why? I uploaded a screenshot about my result here: http://server6.theimagehosting.com/image.php?img=second_fourier.png Bela -- View this message in context: http://www.nabble.com/second-2d-fft-gives-the-same-result-as-fft%2Bifft-tp23945026p23945026.html Sent from the Numpy-discussion mailing list archive at Nabble.com. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] second 2d fft gives the same result as fft+ifft
Sorry, I meant: Im(iFT(FT(f))) = Im(FT^2(f)), Re(iFT(FT(f))) != Re(FT^2(f)) DG --- On Tue, 6/9/09, David Goldsmith d_l_goldsm...@yahoo.com wrote: From: David Goldsmith d_l_goldsm...@yahoo.com Subject: Re: [Numpy-discussion] second 2d fft gives the same result as fft+ifft To: Discussion of Numerical Python numpy-discussion@scipy.org Date: Tuesday, June 9, 2009, 11:01 AM --- On Tue, 6/9/09, Matthieu Brucher matthieu.bruc...@gmail.com wrote: Hi, Is it really ? You only show the imaginary part of the FFT, so you can't be sure of what you are saying. Indeed, is there not a label for a function f which satisfies Im(iFFT(f)) = Im(FFT^2(f)), Re(iFFT(f)) != Re(FFT^2(f))? (And similarly if Im and Re roles are reversed.) Seems like the class of such functions (if any exist) might have some interesting properties... DG Don't forget that the only difference between FFT and iFFT is (besides of teh scaling factor) a minus sign in the exponent. Matthieu 2009/6/9 bela bela.miha...@gmail.com: I tried to calculate the second fourier transformation of an image with the following code below: --- import pylab import numpy ### Create a simple image fx = numpy.zeros( 128**2 ).reshape(128,128).astype( numpy.float ) for i in xrange(8): for j in xrange(8): fx[i*8+16][j*8+16] = 1.0 ### Fourier Transformations Ffx = numpy.copy( numpy.fft.fft2( fx ).real ) # 1st fourier FFfx = numpy.copy( numpy.fft.fft2( Ffx ).real ) # 2nd fourier IFfx = numpy.copy( numpy.fft.ifft2( Ffx ).real ) # inverse fourier ### Display result pylab.figure( 1, figsize=(8,8), dpi=125 ) pylab.subplot(221) pylab.imshow( fx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( fx ) pylab.subplot(222) pylab.imshow( Ffx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( Ffx ) pylab.subplot(223) pylab.imshow( FFfx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( FFfx ) pylab.subplot(224) pylab.imshow( IFfx, cmap=pylab.cm.gray ) pylab.colorbar() pylab.title( IFfx ) pylab.show() --- On my computer FFfx is the same as IFfx. but why? I uploaded a screenshot about my result here: http://server6.theimagehosting.com/image.php?img=second_fourier.png Bela -- View this message in context: http://www.nabble.com/second-2d-fft-gives-the-same-result-as-fft%2Bifft-tp23945026p23945026.html Sent from the Numpy-discussion mailing list archive at Nabble.com. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Multiplying Python float to numpy.array of objects works but fails with a numpy.float64, numpy Bug?
Sun, 07 Jun 2009 12:09:40 +0200, Sebastian Walter wrote: from numpy import * import numpy print 'numpy.__version__=',numpy.__version__ class adouble: def __init__(self,x): self.x = x def __mul__(self,rhs): if isinstance(rhs,adouble): return adouble(self.x * rhs.x) else: return adouble(self.x * rhs) def __str__(self): return str(self.x) [clip] print u * float64(3.)# _NOT_ OK! [clip] Should I open a ticket for that? Please do, the current behavior doesn't seem correct. -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Adding zero-dimension arrays, bug?
On Tue, Jun 9, 2009 at 09:38, David Paul Reichertd.p.reich...@sms.ed.ac.uk wrote: Hi, Numpy let's me define arrays with zero rows and/or columns, and that's wanted behaviour from what I have read in discussions. However, I can add an array with zero rows to an array with one row (but not more), resulting in another zero row array, like so: In: a = zeros((4,0)) In: a Out: array([], shape=(4, 0), dtype=float64) In: b = zeros((4,1)) In: b Out: array([[ 0.], [ 0.], [ 0.], [ 0.]]) In: a + b Out: array([], shape=(4, 0), dtype=float64) Is this a bug? This should give a shape mismatch error, shouldn't? No. According to the rules of broadcasting, the axis will match iff the sizes from each array are the same OR one of them is 1. So (4,1) will broadcast with (4,0) to result in a (4,0) array. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Howto vectorise a dot product ?
Dear all, Can someone point me to a doc on dot product vectorisation ? Here is what I try to do : I've got a rotation function which looks like : def rotat_scal(phi, V): s = math.sin(phi) c = math.cos(phi) M = np.zeros((3, 3)) M[2, 2] = M[1, 1] = c M[1, 2] = -s M[2, 1] = s M[0, 0] = 1 return np.dot(M, V) (where phi is a scalar, and V and array of size (3,1)) Now, I want to apply it to a time series of phi and V, in a vectorised way. So, I tried to simply add a first dimension : Phi is now of size(n) and V (n, 3). (I really whish to have this shape, for direct correspondance to file). The corresponding function looks like : def rotat_vect(phi, V): s = np.sin(phi) c = np.cos(phi) M = np.zeros((len(phi), 3, 3)) M[:, 2, 2] = M[:, 1, 1] = c M[:, 1, 2] = -s M[:, 2, 1] = s M[:, 0, 0] = np.ones (len(phi)) return np.dot(M, V) It was not really a surprise to see that it didn't work : [...] return np.dot(M, V) ValueError: objects are not aligned Any hint ? Bruno. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Inquiry Regarding F2PY Windows Content
Would it be a reasonable request, that the F2PY Windows web page contain known combinations of version numbers for Python, Numpy and Gfortran verified to play nice? Some references as to queried compiler system environmental variables would be useful also. Thanks, Andy ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Howto vectorise a dot product ?
On 9-Jun-09, at 2:56 PM, bruno Piguet wrote: Phi is now of size(n) and V (n, 3). (I really whish to have this shape, for direct correspondance to file). The corresponding function looks like : def rotat_vect(phi, V): s = np.sin(phi) c = np.cos(phi) M = np.zeros((len(phi), 3, 3)) M[:, 2, 2] = M[:, 1, 1] = c M[:, 1, 2] = -s M[:, 2, 1] = s M[:, 0, 0] = np.ones (len(phi)) return np.dot(M, V) Well, if you make V have a singleton dimension on the end you can then do it, but you will get one more axis than you care about. The help for dot() says this: For 2-D arrays it is equivalent to matrix multiplication, and for 1-D arrays to inner product of vectors (without complex conjugation). For N dimensions it is a sum product over the last axis of `a` and the second-to-last of `b`:: dot(a, b)[i,j,k,m] = sum(a[i,j,:] * b[k,:,m]) So changing your code to this: return np.dot(M, V[:,:,np.newaxis])[arange(len(phi)), :, arange(len(phi)), :] will do what you want, but it will also do a lot of useless multiplication in computing that product. I'm not sure of any better way, and am kind of curious myself (since I often have to take products of one or several vectors with several matrices). David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Howto vectorise a dot product ?
On Tue, Jun 9, 2009 at 12:56 PM, bruno Piguet bruno.pig...@gmail.comwrote: Dear all, Can someone point me to a doc on dot product vectorisation ? Here is what I try to do : I've got a rotation function which looks like : def rotat_scal(phi, V): s = math.sin(phi) c = math.cos(phi) M = np.zeros((3, 3)) M[2, 2] = M[1, 1] = c M[1, 2] = -s M[2, 1] = s M[0, 0] = 1 return np.dot(M, V) (where phi is a scalar, and V and array of size (3,1)) Now, I want to apply it to a time series of phi and V, in a vectorised way. So, I tried to simply add a first dimension : Phi is now of size(n) and V (n, 3). (I really whish to have this shape, for direct correspondance to file). Well, in this case you can use complex multiplication and either work with just the x,y components or use two complex components, i.e., [x + 1j*y, z]. In the first case you can then do the rotation as V*exp(1j*phi). If you want more general rotations, a ufunc for quaternions would do the trick. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance matrix multiplication vs. matlab
Hi, I'm one of the Eigen developers and was pointed to your discussion. I just want to clarify a few things for future reference (not trying to get you to use Eigen): No, eigen does not provide a (complete) BLAS/LAPACK interface. True, I don't know if that's even a goal of eigen Not a goal indeed, though there's agreement that such a bridge would be a nice add-on. (Would be a one-directional bridge though. You can't express with BLAS/LAPACK all what you can express with the Eigen API). (it started as a project for KDE, to support high performance core computations for things like spreadsheet and co). Yes, that's how it started 3 years ago. A lot changed since, though. See http://eigen.tuxfamily.org/index.php?title=Main_Page#Projects_using_Eigen Eigen is: - not mature. Fair enough - heavily expression-template-based C++, meaning compilation takes ages No, because _we_ are serious about compilation times, unlike other c++ template libraries. But granted, compilation times are not as short as a plain C library either. + esoteric, impossible to decypher compilation errors. Try it ;) See e.g. this comment: http://www.macresearch.org/interview-eigen-matrix-library#comment-14667 - SSE dependency harcoded, since it is setup at build time. That's going backward IMHO - I would rather see a numpy/scipy which can load the optimized code at runtime. Eigen doesn't _require_ any SIMD instruction set although it can use SSE / AltiVec if enabled. It is true that with Eigen this is set up at build time, but this is only because it is squarely _not_ Eigen's job to do runtime platform checks. Eigen isn't a binary library. If you want a runtime platform switch, just compile your critical Eigen code twice, one with SSE one without, and do the platform check in your own app. The good thing here is that Eigen makes sure that the ABI is independent of whether vectorization is enabled. And to reply to Matthieu's mail: I would add that it relies on C++ compiler extensions (the restrict keyword) as does blitz. You unfortunately can't expect every compiler to support it unless the C++ committee finally adds it to the standard. currently we have: #define EIGEN_RESTRICT __restrict This is ready to be replaced by an empty symbol if some compiler doesn't support restrict. The only reason why we didn't do this is that all compilers we've encountered so far support restrict. Cheers, Benoit ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance matrix multiplication vs. matlab
On Tue, Jun 9, 2009 at 7:46 PM, Benoit Jacob jacob.benoi...@gmail.comwrote: Hi, I'm one of the Eigen developers and was pointed to your discussion. I just want to clarify a few things for future reference (not trying to get you to use Eigen): No, eigen does not provide a (complete) BLAS/LAPACK interface. True, I don't know if that's even a goal of eigen Not a goal indeed, though there's agreement that such a bridge would be a nice add-on. (Would be a one-directional bridge though. You can't express with BLAS/LAPACK all what you can express with the Eigen API). (it started as a project for KDE, to support high performance core computations for things like spreadsheet and co). Yes, that's how it started 3 years ago. A lot changed since, though. See http://eigen.tuxfamily.org/index.php?title=Main_Page#Projects_using_Eigen Eigen is: - not mature. Fair enough - heavily expression-template-based C++, meaning compilation takes ages No, because _we_ are serious about compilation times, unlike other c++ template libraries. But granted, compilation times are not as short as a plain C library either. I wonder if it is possible to have a compiler/parser that does nothing but translate templates into c? Say, something written in python ;) Name mangling would be a problem but could perhaps be simplified for the somewhat limited space needed for numpy/scipy. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance matrix multiplication vs. matlab
Hi Benoit, Benoit Jacob wrote: No, because _we_ are serious about compilation times, unlike other c++ template libraries. But granted, compilation times are not as short as a plain C library either. I concede it is not as bad as the heavily templated libraries in boost. But C++ is just horribly slow to compile, at least with g++ - in scipy, half of the compilation time is spent for a couple of C++ files which uses simple templates. And the compiler takes a lot of memory during compilation (~ 300 Mb per file - that's a problem because I rely a lot on VM to build numpy/scipy binaries). Eigen doesn't _require_ any SIMD instruction set although it can use SSE / AltiVec if enabled. If SSE is not enabled, my (very limited) tests show that eigen does not perform as well as a stock debian ATLAS on the benchmarks given by eigen. For example: g++ benchBlasGemm.cpp -I .. -lblas -O2 -DNDEBUG ./a.out 300 cblas: 0.034222 (0.788 GFlops/s) eigen : 0.0863581 (0.312 GFlops/s) eigen : 0.121259 (0.222 GFlops/s) g++ benchBlasGemm.cpp -I .. -lblas -O2 -DNDEBUG -msse2 ./a.out 300 cblas: 0.035438 (0.761 GFlops/s) eigen : 0.0182271 (1.481 GFlops/s) eigen : 0.0860961 (0.313 GFlops/s) (on a PIV, which may not be very representative of current architectures) It is true that with Eigen this is set up at build time, but this is only because it is squarely _not_ Eigen's job to do runtime platform checks. Eigen isn't a binary library. If you want a runtime platform switch, just compile your critical Eigen code twice, one with SSE one without, and do the platform check in your own app. The good thing here is that Eigen makes sure that the ABI is independent of whether vectorization is enabled. I understand that it is not a goal of eigen, and that should be the application's job. It is just that MKL does it automatically, and doing it in a cross platform way in the context of python extensions is quite hard because of various linking strategies on different OS. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Inquiry Regarding F2PY Windows Content
Carl, Andrew F (AS) wrote: Would it be a reasonable request, that the F2PY Windows web page contain known combinations of version numbers for Python, Numpy and Gfortran verified to play nice? Some references as to queried compiler system environmental variables would be useful also. I have added some numpy.distutils support for gfortran on windows, but Windows + gfortran + numpy is unlikely to work well unless you build numpy by yourself with gfortran. I have actually been considering a move to gfortran for windows builds, but I would prefer waiting mingw to officially support for gcc 4.* series. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Definitions of pv, fv, nper, pmt, and rate
On Tue, Jun 9, 2009 at 12:58 PM, David Goldsmithd_l_goldsm...@yahoo.com wrote: --- On Tue, 6/9/09, Skipper Seabold jsseab...@gmail.com wrote: These are the two most basic building blocks of time value problems, discounting one cash flow and an annuity. There are *plenty* of examples and use cases for uneven cash flows or for providing a given pv or fv. Without even getting into actual financial contracts, suppose I have an investment account that already has $10,000 and I plan to add $500 every month and earn 4%. Then we would need something like fv to tell me how much this will be worth after 180 months. I don't necessarily need a reference to tell me this would be useful to know. Use case examples aren't the problem; worked out examples combining these two principles aren't the problem; usefulness isn't the problem; the problem is one of meeting a particular reference standard. I don't know that these are formulas per se, rather than Except that we do provide a formula in our help doc; perhaps the solution is to get rid of that and include an explanation of how our function combines the two basic formulae (for which I do have a hard-copy reference: Gitman, L. J., 2003. Principals of Managerial Finance (Brief), 3rd Ed. Pearson Education) to handle the more general case. FV (even if one of these last two values is zero). If you need a textbook reference, as I said before you could literally pick up any corporate finance text and derive these functions from the basics. I don't question that, what I question is the appropriateness of such derivation in numpy's help doc; as I see it, one of the purposes of providing references in our situation is precisely to avoid having to include derivations in our help doc. But it's all moot, as Robert has furnished me with a reference specifically for the formula we do have, and, although it's an electronic reference, it seems stable enough to warrant an exception to the references should be hard-copy AMAP policy. Just to follow up a bit with this. I don't want to beat a dead horse, but I'd just like to share my experience with these documents and functions so far. I have implemented the ipmt and ppmt functions that were not implemented in numpy.lib.financial as well as written some tests. ipmt is one of the functions where there was a discrepancy between what OO and Excel report for the beginning of period payment assumptions and what Gnumeric and Kspread report as stated in the OpenFormula document referenced above (I discovered errors in this document as well as the openoffice documents referenced btw but they become obvious when you work these problems out). OpenFormula lists the Gnumeric/Kspread as the result in the document, but there is still a question to which is correct. Well, I was able to derive both results, and as I suspected the Gnumeric/Kspread was based on an incorrect assumption (or a mistake in implementation) not a different formula. My point with the derivations wasn't to include them in the documentation, but rather to find out what assumptions are being made and then deduce which results are correct. In the cases of these simple spreadsheet functions I think it should be obvious if it's right or wrong. If there is any interest in adding the ipmt and ppmt functions now, I can apply a patch. If not, I can keep them separate as I work on the other functions. Cheers, Skipper ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Jcamp dx format
On 7-Jun-09, at 4:56 AM, giorgio.luci...@inwind.it wrote: Sorry for cross posting Hello to all, I've done a script for importing all spectra files in a directory and merge all them in one matrix. The file imported are dx files. the bad part is that the file is in matlab and it requite a function from bioinformatic toolbox (jcamp read). And now I just wnat to do the same in python. I guess I will have no problem for translating the script but I think I dont' have the time (and capabilities) to rewrite something like jcampread. Since jcamp dx format it's quite common among scientist. Does anyone can share some script/function for importing them in python (I guess that also a r routine can do the trick but I will prefer to use python). Thanks in advance to all Giorgio Googling revealed this which should do the trick: http://code.google.com/p/pyms/ David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] performance matrix multiplication vs. matlab
David Warde-Farley wrote: On 9-Jun-09, at 3:54 AM, David Cournapeau wrote: For example, what ML people call PCA is called Karhunen Loéve in signal processing, and the concepts are quite similar. Yup. This seems to be a nice set of review notes: http://www.ece.rutgers.edu/~orfanidi/ece525/svd.pdf This looks indeed like a very nice review from a signal processing approach. I never took the time to understand the similarities/differences/connections between traditional SP approaches and the machine learning approach. I wonder if the subspaces methods ala PENCIL/MUSIC and co have a (useful) interpretation in a more ML approach, I never really thought about it. I guess other people had :) And going further than just PCA/KLT, tying it together with maximum likelihood factor analysis/linear dynamical systems/hidden Markov models, http://www.cs.toronto.edu/~roweis/papers/NC110201.pdf As much as I like this paper, I always felt that you miss a lot of insights when considering PCA only from a purely statistical POV. I really like the consideration of PCA within a function approximation POV (the chapter 9 of the Mallat book on wavelet is cristal clear, for example, and it is based on all those cool functional spaces theory likes Besov space). cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Definitions of pv, fv, nper, pmt, and rate
--- On Tue, 6/9/09, Skipper Seabold jsseab...@gmail.com wrote: I have implemented the ipmt and ppmt functions that were not implemented in numpy.lib.financial as well as written some tests. Thanks! ipmt is one of the functions where there was a discrepancy between what OO and Excel report for the beginning of period payment assumptions and what Gnumeric and Kspread report as stated in the OpenFormula document referenced above (I discovered errors in this document as well as the openoffice documents referenced btw but they become obvious when you work these problems out). And the nightmare worsens (IMO). OpenFormula lists the Gnumeric/Kspread as the result in the document, but there is still a question to which is correct. Well, I was able to derive both results, and as I suspected the Gnumeric/Kspread was based on an incorrect assumption (or a mistake in implementation) not a different formula. My point with the derivations wasn't to include them in the documentation, but rather to find out what assumptions are being made and then deduce which results are correct. In the cases of these simple spreadsheet functions I think it should be obvious if it's right or wrong. OK, I concede defeat: if it is the wisdom of the PTB that numpy.financial be retained, I will stop messing w/ their help doc, 'cause I'm clearly in over my head. DG If there is any interest in adding the ipmt and ppmt functions now, I can apply a patch. If not, I can keep them separate as I work on the other functions. Cheers, Skipper ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Question about memmap
Hello,
I am having problem while trying to memory map a simple file (attached as
test.txt)
In IPython
data = memmap('test.txt', mode='r', dtype=double, shape=(3,5))
data
memmap([[ 3.45616501e-86, 4.85780149e-33, 4.85787493e-33,
5.07185821e-86, 4.85780159e-33],
[ 4.85787493e-33, 5.07185821e-86, 1.28444278e-57,
1.39804066e-76, 4.85787506e-33],
[ 4.83906715e-33, 4.85784273e-33, 4.85787506e-33,
4.83906715e-33, 4.85784273e-33]])
which is not what is in the file. Tried different dtype float options, but
always the same result.
Could you tell me what could be wrong?
Thanks...
Gökhan
42501. 99. 99. 99. 99.
42502. 99. 99. 99. 99.
42503. 99. 99. 99. 99.___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about memmap
Hi, I am having problem while trying to memory map a simple file (attached as test.txt) The file looks like a text file, but memmap is for binary files. Could that be the problem? Best, Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about memmap
On Wed, Jun 10, 2009 at 12:34 AM, Matthew Brett matthew.br...@gmail.comwrote: Hi, I am having problem while trying to memory map a simple file (attached as test.txt) The file looks like a text file, but memmap is for binary files. Could that be the problem? Best, Matthew I don't see such a restriction in memmap function based on its help at or memmap? from IPython. http://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html#numpy.memmap I thought it will happily work with text files too :( Gökhan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about memmap
--- On Tue, 6/9/09, Gökhan SEVER gokhanse...@gmail.com wrote: Matthew Brett matthew.br...@gmail.com wrote: I am having problem while trying to memory map a simple file (attached as test.txt) The file looks like a text file, but memmap is for binary files. Could that be the problem? Matthew I don't see such a restriction in memmap function based on its help Fixed (at least in the Numpy Doc Wiki, don't know how long it will take for that to propagate to a release) at or memmap? from IPython. Sorry, no authority over there. I thought it will happily work with text files too :( Not the soln. you were hoping for, I know, sorry. DG ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about memmap
Sorry for the double post, my link and/or browser was acting up. DG ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
