[Numpy-discussion] How to make lazy derived arrays in a recarray view of a memmap on large files

2009-01-16 Thread Kim Hansen
Hi numpy forum I need to efficiently handle some large (300 MB) recordlike binary files, where some data fields are less than a byte and thus cannot be mapped in a record dtype immediately. I would like to be able to access these derived arrays in a memory efficient manner but I cannot figure

[Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-16 Thread Francesc Alted
Announcing Numexpr 1.1 Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like 3*a+4*b) are accelerated and use less memory than doing the same calculation in Python. The expected speed-ups

Re: [Numpy-discussion] remove need for site.cfg on default

2009-01-16 Thread Darren Dale
On Thu, Jan 15, 2009 at 11:53 PM, David Cournapeau courn...@gmail.comwrote: On Fri, Jan 16, 2009 at 12:18 AM, Darren Dale dsdal...@gmail.com wrote: Hi Jarrod, On Wed, Jan 14, 2009 at 2:21 AM, Jarrod Millman mill...@berkeley.edu wrote: Due to the fact that I was tired of adding

Re: [Numpy-discussion] error handling with f2py?

2009-01-16 Thread Pearu Peterson
On Thu, January 15, 2009 6:17 pm, Sturla Molden wrote: Is it possible to make f2py raise an exception if a fortran routine signals an error? If I e.g. have subroutine foobar(a, ierr) Can I get an exception automatically raised if ierr != 0? Yes, for that you need to provide your own

Re: [Numpy-discussion] Singular Matrix problem with Matplit lib in Numpy (Windows - AMD64)

2009-01-16 Thread George
Hello. I am terribly sorry. I was mistaken last night. I had the latest Matplotlib version 0.98.5.2 and I thought the bug was fixed but it hasn't. Let me explain. In the file MPL_isnan.h line 26 there is a declaration: typedef long int MPL_Int64 This is fine for

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-16 Thread Gregor Thalhammer
Francesc Alted schrieb: Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like 3*a+4*b) are accelerated and use less memory than doing the same calculation in Python. The expected speed-ups for Numexpr respect to NumPy are between 0.95x

Re: [Numpy-discussion] numpy.testing.asserts and masked array

2009-01-16 Thread Pierre GM
On Jan 16, 2009, at 10:51 AM, josef.p...@gmail.com wrote: I have a regression result with masked arrays that produces a masked array output, estm5.yhat, and I want to test equality to the benchmark case, estm1.yhat, with the asserts in numpy.testing, but I am getting strange results. ...

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-16 Thread Francesc Alted
A Friday 16 January 2009, j...@physics.ucf.edu escrigué: Hi Francesc, Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like 3*a+4*b) are accelerated and use less memory than doing the same calculation in Python. Please pardon my

Re: [Numpy-discussion] error handling with f2py?

2009-01-16 Thread Sturla Molden
On 1/16/2009 2:16 PM, Pearu Peterson wrote: Yes, for that you need to provide your own fortran call code using f2py callstatement construct. The initial fortran call code can be obtained from f2py generated modulenamemodule.c file, for instance. Thank you, Pearu :) f2py is really a

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-16 Thread Sebastian Haase
Hi Francesc, this is a wonderful project ! I was just wondering if you would / could support single precision float arrays ? In 3+D image analysis we generally don't have enough memory to effort double precision; and we could save our selves lots of extra C coding (or Cython) coding of we could

Re: [Numpy-discussion] numpy.testing.asserts and masked array

2009-01-16 Thread josef . pktd
On Fri, Jan 16, 2009 at 10:59 AM, Pierre GM pgmdevl...@gmail.com wrote: On Jan 16, 2009, at 10:51 AM, josef.p...@gmail.com wrote: I have a regression result with masked arrays that produces a masked array output, estm5.yhat, and I want to test equality to the benchmark case, estm1.yhat, with

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-16 Thread Francesc Alted
A Friday 16 January 2009, Gregor Thalhammer escrigué: I also gave a try to the vector math library (VML), contained in Intel's Math Kernel Library. This offers a fast implementation of mathematical functions, operating on array. First I implemented a C extension, providing new ufuncs. This

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-16 Thread Francesc Alted
A Friday 16 January 2009, Sebastian Haase escrigué: Hi Francesc, this is a wonderful project ! I was just wondering if you would / could support single precision float arrays ? As I said before, it is doable, but I don't know if I will have time enough to implement this myself. In 3+D image

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-16 Thread Dag Sverre Seljebotn
Francesc Alted wrote: A Friday 16 January 2009, j...@physics.ucf.edu escrigué: Right now, I'm not quite sure whether the problem you are solving is merely the case of expressions-in-strings, and there is no advantage for expressions-in-code, or whether your expressions-in-strings are faster

Re: [Numpy-discussion] Singular Matrix problem with Matplitlib in Numpy (Windows - AMD64)

2009-01-16 Thread Andrew Straw
John Hunter wrote: Andrew, since you are the original author of the isnan port, could you patch the branch and the trunk to take care of this? Done in r6791 and r6792. Sorry for the trouble. Now I just hope we don't get a problem with long long, although now if _ISOC99_SOURCE is defined,

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-16 Thread Gregor Thalhammer
Francesc Alted schrieb: A Friday 16 January 2009, Gregor Thalhammer escrigué: I also gave a try to the vector math library (VML), contained in Intel's Math Kernel Library. This offers a fast implementation of mathematical functions, operating on array. First I implemented a C extension,

Re: [Numpy-discussion] Help with interpolating missing values from a 3D scanner

2009-01-16 Thread David Bolme
Thanks for all the ideas. I think I will look into the scikits.delaunay, Rbf, or gaussian smoothing approach. My best idea is similar to the Gaussian smoothing. Anyway, all of the missing data gaps seem to be small enough that I expect any of these methods to accomplish my purpose.

[Numpy-discussion] Please don't use google code for hosting

2009-01-16 Thread Matthew Brett
Hi, I am just visiting colleagues in the Cuban Neuroscience Center, and of course I'm trying to persuade them that Python and open-source are the way forward. This is made more difficult because several projects - for example pyglet - have their repositories on Google code. Google, unlike any

Re: [Numpy-discussion] Please don't use google code for hosting

2009-01-16 Thread Kevin Jacobs jac...@bioinformed.com
On Fri, Jan 16, 2009 at 7:07 PM, Matthew Brett matthew.br...@gmail.comwrote: So, please, if you are considering google code for hosting, consider other options. Seems odd that you'd post that from a gmail account. I do sympathize with your suggestion, but I don't have a better alternative to

Re: [Numpy-discussion] Please don't use google code for hosting

2009-01-16 Thread Alan Jackson
On Fri, 16 Jan 2009 19:24:56 -0500 Kevin Jacobs jac...@bioinformed.com bioinfor...@gmail.com wrote: On Fri, Jan 16, 2009 at 7:07 PM, Matthew Brett matthew.br...@gmail.comwrote: So, please, if you are considering google code for hosting, consider other options. Seems odd that you'd

Re: [Numpy-discussion] Please don't use google code for hosting

2009-01-16 Thread Robert Kern
On Fri, Jan 16, 2009 at 18:07, Matthew Brett matthew.br...@gmail.com wrote: Hi, I am just visiting colleagues in the Cuban Neuroscience Center, and of course I'm trying to persuade them that Python and open-source are the way forward. This is made more difficult because several projects -

[Numpy-discussion] tensor contractions

2009-01-16 Thread Gideon Simpson
Suppose I have a 3d array, A, with dimensions 2 x 2 x N, and a 2d 2 x N array, u. I interpret A as N 2x2 matrices and u as N 2d vectors. Suppose I want to apply the mth matrix to the mth vector, i.e. A[, , m] u[, m] = v[, m] Aside from doing A[0,0,:] u[0,:] + A[0,1,:] u[1,:] = v[0,:] and

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-16 Thread Ted Horst
Note that Apple has a similar library called vForce: http://developer.apple.com/ReleaseNotes/Performance/RN-vecLib/index.html http://developer.apple.com/documentation/Performance/Conceptual/vecLib/Reference/reference.html I think these libraries use several techniques and are not

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-16 Thread Olivier Grisel
2009/1/16 Gregor Thalhammer gregor.thalham...@gmail.com: Francesc Alted schrieb: Wow, pretty nice speed-ups indeed! In fact I was thinking in including support for threading in Numexpr (I don't think it would be too difficult, but let's see). BTW, do you know how VML is able to achieve a