Re: [Numpy-discussion] automatically avoiding temporary arrays

2017-02-27 Thread Matthew Brett
Hi, On Mon, Feb 27, 2017 at 11:46 AM, Benjamin Root wrote: > Ah, that wheel would be a huge help. Most of the packages I have as > dependencies for this project were compiled against v1.10, so I am hoping > that there won't be too big of a problem. I don't think so - the problems generally arise

Re: [Numpy-discussion] automatically avoiding temporary arrays

2017-02-27 Thread Benjamin Root
Ah, that wheel would be a huge help. Most of the packages I have as dependencies for this project were compiled against v1.10, so I am hoping that there won't be too big of a problem. Are the mostlinux wheels still compatible with CentOS5, or has the base image been bumped to CentOS6? Cheers! Ben

Re: [Numpy-discussion] automatically avoiding temporary arrays

2017-02-27 Thread Matthew Brett
Hi, On Mon, Feb 27, 2017 at 11:27 AM, Charles R Harris wrote: > > > On Mon, Feb 27, 2017 at 11:43 AM, Benjamin Root > wrote: >> >> What's the timeline for the next release? I have the perfect usecase for >> this (Haversine calculation on large arrays that takes up ~33% of one of my >> processing

Re: [Numpy-discussion] automatically avoiding temporary arrays

2017-02-27 Thread Charles R Harris
On Mon, Feb 27, 2017 at 11:43 AM, Benjamin Root wrote: > What's the timeline for the next release? I have the perfect usecase for > this (Haversine calculation on large arrays that takes up ~33% of one of my > processing scripts). However, to test it out, I have a huge dependency mess > to wade t

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-06 Thread srean
On Wed, Oct 5, 2016 at 5:36 PM, Robert McLeod wrote: > > It's certainly true that numexpr doesn't create a lot of OP_COPY > operations, rather it's optimized to minimize them, so probably it's fewer > ops than naive successive calls to numpy within python, but I'm unsure if > there's any differen

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-05 Thread Robert McLeod
On Wed, Oct 5, 2016 at 1:11 PM, srean wrote: > Thanks Francesc, Robert for giving me a broader picture of where this fits > in. I believe numexpr does not handle slicing, so that might be another > thing to look at. > Dereferencing would be relatively simple to add into numexpr, as it would jus

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-05 Thread srean
Thanks Francesc, Robert for giving me a broader picture of where this fits in. I believe numexpr does not handle slicing, so that might be another thing to look at. On Wed, Oct 5, 2016 at 4:26 PM, Robert McLeod wrote: > > As Francesc said, Numexpr is going to get most of its power through > gr

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-05 Thread Robert McLeod
All, On Wed, Oct 5, 2016 at 11:46 AM, Francesc Alted wrote: > 2016-10-05 8:45 GMT+02:00 srean : > >> Good discussion, but was surprised by the absence of numexpr in the >> discussion., given how relevant it (numexpr) is to the topic. >> >> Is the goal to fold in the numexpr functionality (and be

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-03 Thread Marten van Kerkwijk
Note that numpy does store some larger arrays already, in the fft module. (In fact, this was a cache of unlimited size until #7686.) It might not be bad if the same cache were used more generally. That said, if newer versions of python are offering ways of doing this better, maybe that is the best

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-03 Thread Pauli Virtanen
Mon, 03 Oct 2016 15:07:28 -0400, Benjamin Root kirjoitti: > With regards to arguments about holding onto large arrays, I would like > to emphasize that my original suggestion mentioned weakref'ed numpy > arrays. > Essentially, the idea is to claw back only the raw memory blocks during > that limbo

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-03 Thread Benjamin Root
With regards to arguments about holding onto large arrays, I would like to emphasize that my original suggestion mentioned weakref'ed numpy arrays. Essentially, the idea is to claw back only the raw memory blocks during that limbo period between discarding the numpy array python object and when pyt

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-03 Thread Julian Taylor
On 03.10.2016 20:23, Chris Barker wrote: > > > On Mon, Oct 3, 2016 at 3:16 AM, Julian Taylor > mailto:jtaylor.deb...@googlemail.com>> > wrote: > > the problem with this approach is that we don't really want numpy > hogging on to hundreds of megabytes of memory by default so it would >

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-03 Thread Chris Barker
On Mon, Oct 3, 2016 at 3:16 AM, Julian Taylor wrote: > the problem with this approach is that we don't really want numpy > hogging on to hundreds of megabytes of memory by default so it would > need to be a user option. > indeed -- but one could set an LRU cache to be very small (few items, not

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-02 Thread Hush Hush
. (David Cournapeau) >3. Re: Dropping sourceforge for releases. (Vincent Davis) > > > -- > > Message: 1 > Date: Sun, 2 Oct 2016 09:10:45 -0400 > From: Benjamin Root > To: Discussion of Numerical Python > Subj

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-09-30 Thread Julian Taylor
On 30.09.2016 23:09, josef.p...@gmail.com wrote: > On Fri, Sep 30, 2016 at 9:38 AM, Julian Taylor > wrote: >> hi, >> Temporary arrays generated in expressions are expensive as the imply >> extra memory bandwidth which is the bottleneck in most numpy operations. >> For example: >> >> r = a + b + c