Re: [Numpy-discussion] caching large allocations on gnu/linux

2017-03-14 Thread Julian Taylor
On 13.03.2017 19:54, Francesc Alted wrote: > 2017-03-13 18:11 GMT+01:00 Julian Taylor >: > > On 13.03.2017 16:21, Anne Archibald wrote: > > > > > > On Mon, Mar 13, 2017 at 12:21 PM Julian Taylor > >

Re: [Numpy-discussion] caching large allocations on gnu/linux

2017-03-13 Thread Francesc Alted
2017-03-13 18:11 GMT+01:00 Julian Taylor : > On 13.03.2017 16:21, Anne Archibald wrote: > > > > > > On Mon, Mar 13, 2017 at 12:21 PM Julian Taylor > > > > > wrote: > > > > Should it be agreed

Re: [Numpy-discussion] caching large allocations on gnu/linux

2017-03-13 Thread Julian Taylor
On 13.03.2017 16:21, Anne Archibald wrote: > > > On Mon, Mar 13, 2017 at 12:21 PM Julian Taylor > > > wrote: > > Should it be agreed that caching is worthwhile I would propose a very > simple implementation. We only

Re: [Numpy-discussion] caching large allocations on gnu/linux

2017-03-13 Thread Anne Archibald
On Mon, Mar 13, 2017 at 12:21 PM Julian Taylor < jtaylor.deb...@googlemail.com> wrote: Should it be agreed that caching is worthwhile I would propose a very > simple implementation. We only really need to cache a small handful of > array data pointers for the fast allocate deallocate cycle that

[Numpy-discussion] caching large allocations on gnu/linux

2017-03-13 Thread Julian Taylor
Hi, As numpy often allocates large arrays and one factor in its performance is faulting memory from the kernel to the process. This has some cost that is relatively significant. For example in this operation on large arrays it accounts for 10-15% of the runtime: import numpy as np a =