Re: [Numpy-discussion] automatically avoiding temporary arrays

2017-02-27 Thread Matthew Brett
Hi, On Mon, Feb 27, 2017 at 11:46 AM, Benjamin Root wrote: > Ah, that wheel would be a huge help. Most of the packages I have as > dependencies for this project were compiled against v1.10, so I am hoping > that there won't be too big of a problem. I don't think so - the

Re: [Numpy-discussion] automatically avoiding temporary arrays

2017-02-27 Thread Benjamin Root
Ah, that wheel would be a huge help. Most of the packages I have as dependencies for this project were compiled against v1.10, so I am hoping that there won't be too big of a problem. Are the mostlinux wheels still compatible with CentOS5, or has the base image been bumped to CentOS6? Cheers! Ben

Re: [Numpy-discussion] automatically avoiding temporary arrays

2017-02-27 Thread Matthew Brett
Hi, On Mon, Feb 27, 2017 at 11:27 AM, Charles R Harris wrote: > > > On Mon, Feb 27, 2017 at 11:43 AM, Benjamin Root > wrote: >> >> What's the timeline for the next release? I have the perfect usecase for >> this (Haversine calculation on large

Re: [Numpy-discussion] automatically avoiding temporary arrays

2017-02-27 Thread Charles R Harris
On Mon, Feb 27, 2017 at 11:43 AM, Benjamin Root wrote: > What's the timeline for the next release? I have the perfect usecase for > this (Haversine calculation on large arrays that takes up ~33% of one of my > processing scripts). However, to test it out, I have a huge

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-06 Thread srean
On Wed, Oct 5, 2016 at 5:36 PM, Robert McLeod wrote: > > It's certainly true that numexpr doesn't create a lot of OP_COPY > operations, rather it's optimized to minimize them, so probably it's fewer > ops than naive successive calls to numpy within python, but I'm unsure if

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-05 Thread Robert McLeod
On Wed, Oct 5, 2016 at 1:11 PM, srean wrote: > Thanks Francesc, Robert for giving me a broader picture of where this fits > in. I believe numexpr does not handle slicing, so that might be another > thing to look at. > Dereferencing would be relatively simple to add into

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-05 Thread srean
Thanks Francesc, Robert for giving me a broader picture of where this fits in. I believe numexpr does not handle slicing, so that might be another thing to look at. On Wed, Oct 5, 2016 at 4:26 PM, Robert McLeod wrote: > > As Francesc said, Numexpr is going to get most of

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-05 Thread Robert McLeod
All, On Wed, Oct 5, 2016 at 11:46 AM, Francesc Alted wrote: > 2016-10-05 8:45 GMT+02:00 srean : > >> Good discussion, but was surprised by the absence of numexpr in the >> discussion., given how relevant it (numexpr) is to the topic. >> >> Is the goal to

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-03 Thread Marten van Kerkwijk
Note that numpy does store some larger arrays already, in the fft module. (In fact, this was a cache of unlimited size until #7686.) It might not be bad if the same cache were used more generally. That said, if newer versions of python are offering ways of doing this better, maybe that is the

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-03 Thread Pauli Virtanen
Mon, 03 Oct 2016 15:07:28 -0400, Benjamin Root kirjoitti: > With regards to arguments about holding onto large arrays, I would like > to emphasize that my original suggestion mentioned weakref'ed numpy > arrays. > Essentially, the idea is to claw back only the raw memory blocks during > that limbo

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-03 Thread Benjamin Root
With regards to arguments about holding onto large arrays, I would like to emphasize that my original suggestion mentioned weakref'ed numpy arrays. Essentially, the idea is to claw back only the raw memory blocks during that limbo period between discarding the numpy array python object and when

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-03 Thread Julian Taylor
On 03.10.2016 20:23, Chris Barker wrote: > > > On Mon, Oct 3, 2016 at 3:16 AM, Julian Taylor > > > wrote: > > the problem with this approach is that we don't really want numpy > hogging on to hundreds of megabytes of

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-03 Thread Chris Barker
On Mon, Oct 3, 2016 at 3:16 AM, Julian Taylor wrote: > the problem with this approach is that we don't really want numpy > hogging on to hundreds of megabytes of memory by default so it would > need to be a user option. > indeed -- but one could set an LRU cache

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-10-02 Thread Hush Hush
> > To: Discussion of Numerical Python <numpy-discussion@scipy.org> > Subject: Re: [Numpy-discussion] automatically avoiding temporary > arrays > Message-ID: > <CANNq6Fn9eGOnrSGz8Duo8J9oTe3N6xTCSqDDP7nRyNqjFKpAjQ@mail. > gmail.com> > Content-Type

Re: [Numpy-discussion] automatically avoiding temporary arrays

2016-09-30 Thread Julian Taylor
On 30.09.2016 23:09, josef.p...@gmail.com wrote: > On Fri, Sep 30, 2016 at 9:38 AM, Julian Taylor > wrote: >> hi, >> Temporary arrays generated in expressions are expensive as the imply >> extra memory bandwidth which is the bottleneck in most numpy operations. >>