[Numpy-discussion] Memory allocation of arrays and tracemalloc

Hello, I see that the PyDataMem_* APIs call malloc()/free()/etc. directly, instead of going through PyMem_Malloc, etc. This means the memory allocated by those APIs won't be seen by tracemalloc. Is it deliberate? Am I missing something? Regards Antoine.

[Numpy-discussion] Aligned array allocations

Hello, In https://github.com/numpy/numpy/issues/5312 there's a request for an aligned allocator in Numpy (more than the default alignment of the platform's memory allocator). The reason is that on modern vectorization instruction sets, a certain alignment is required for optimal performance

[Numpy-discussion] Aligned / configurable memory allocation

Hello, I apologize for pinging the list, but I was wondering if there was interest in either of https://github.com/numpy/numpy/pull/5457 (make array data aligned by default) or https://github.com/numpy/numpy/pull/5470 (make the array data allocator configurable)? Regards Antoine.

Re: [Numpy-discussion] ANN: numexpr 2.4.3 released

On Mon, 27 Apr 2015 19:35:51 -0400 Neil Girdhar mistersh...@gmail.com wrote: Also, FYI: http://numba.pydata.org/numba-doc/0.6/doc/modules/transforms.html It appears that numba does get the ast similar to pyautodiff and only get the ast from source code as a fallback? That documentation is

Re: [Numpy-discussion] floats for indexing, reshape - too strict ?

On Thu, 02 Jul 2015 08:40:13 -0400 Neal Becker ndbeck...@gmail.com wrote: I'd be concerned that checking each index for exactness would be costly. I'm also concerned that using floats for an index is frequently a mistake and that a warning is what I want. Or just follow Python: Python 3.4.3

[Numpy-discussion] Rationale for integer promotion rules

Hi, We were discussion integer promotion rules amongst the Numba team, and we were wondering about the rationale for Numpy's rules. For example, adding int8 and int8 will give int8 as result (with potential magnitude loss), while adding int8 and uint8 will give int16 as result (promoting to the

Re: [Numpy-discussion] Rationale for integer promotion rules

On Thu, 16 Jul 2015 19:19:58 +0100 Robert Kern robert.k...@gmail.com wrote: On Thu, Jul 16, 2015 at 7:14 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Jul 16, 2015 at 9:18 AM, Antoine Pitrou solip...@pitrou.net wrote: while adding int8 and uint8 will give int16 as result

Re: [Numpy-discussion] Rationale for integer promotion rules

On Thu, 16 Jul 2015 11:14:10 -0700 Nathaniel Smith n...@pobox.com wrote: Also, it is set to stay like this, or will it evolve in the future? I don't know -- if you make a good case for something better then maybe? No, I was just wondering if it would be a good use of our time to try to use

Re: [Numpy-discussion] py2/py3 pickling

On Tue, 25 Aug 2015 19:12:30 +0300 Pauli Virtanen p...@iki.fi wrote: 25.08.2015, 01:15, Chris Laumann kirjoitti: Would it be possible then (in relatively short order) to create a py2 - py3 numpy pickle converter? You probably need to modify the pickle stream directly, replacing *STRING

Re: [Numpy-discussion] Should we drop support for "one file" compilation mode?

On Tue, 6 Oct 2015 11:00:30 +0100 David Cournapeau wrote: > > Assuming one of the rumour is related to some comments I made some time > (years ?) earlier, the context was the ability to hide exported symbols. As > you know, the issue is not to build extensions w/ multiple

Re: [Numpy-discussion] Notes from the numpy dev meeting at scipy 2015

On Wed, 26 Aug 2015 16:45:51 + (UTC) Irwin Zaid iz...@continuum.io wrote: So, we see DyND is having a twofold purpose. The first is to expand upon the kinds of data that NumPy can represent and do computations upon. The second is to provide a standard array package that can cross the

[Numpy-discussion] SHA256 mismatch on SourceForge downloads

Hello, The SourceForge download page for 1.10.0b1 mentions: 89e467cec774527dd254c1e039801726db1367433053801f0d8bc68deac74009 numpy-1.10.0b1.tar.gz But after downloading the file I get: $ sha256sum numpy-1.10.0b1.tar.gz 855695405092686264dc8ce7b3f5c939a6cf1a5639833e841a5bb6fb799cd6a8

Re: [Numpy-discussion] Notes from the numpy dev meeting at scipy 2015

On Tue, 25 Aug 2015 03:03:41 -0700 Nathaniel Smith n...@pobox.com wrote: Supporting third-party dtypes ~ [...] Some features that would become straightforward to implement (e.g. even in third-party libraries) if this were fixed: - missing value support

Re: [Numpy-discussion] 1.10.0rc1

On Tue, 25 Aug 2015 10:26:02 -0600 Charles R Harris charlesr.har...@gmail.com wrote: Hi All, The silence after the 1.10 beta has been eerie. Consequently, I'm thinking of making a first release candidate this weekend. If you haven't yet tested the beta, please do so. It would be good to

Re: [Numpy-discussion] 1.10.0rc1

26, 2015 at 7:31 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, Aug 26, 2015 at 7:11 AM, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 25 Aug 2015 10:26:02 -0600 Charles R Harris charlesr.har...@gmail.com wrote: Hi All, The silence after the 1.10 beta

Re: [Numpy-discussion] 1.10.0rc1

: On Wed, Aug 26, 2015 at 7:31 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, Aug 26, 2015 at 7:11 AM, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 25 Aug 2015 10:26:02 -0600 Charles R Harris charlesr.har...@gmail.com wrote: Hi All, The silence after

Re: [Numpy-discussion] np.sign and object comparisons

On Mon, 31 Aug 2015 10:23:10 -0700 Stephan Hoyer wrote: > > My inclination is that return NaN would be the appropriate choice. It's > certainly consistent with the behavior for float dtypes -- my expectation > for object dtype behavior is that it works exactly like applying the

Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

On Mon, 14 Sep 2015 01:32:13 -0700 Matthew Brett wrote: > >> > >> Does anyone object to me uploading a wheel built from this patched > >> version to pypi as 1.9.2 for Python 3.5 on OSX? It would help to get > >> the ball rolling for Python 3.5 binary wheels. > > > > Why

Re: [Numpy-discussion] [Python-ideas] Should our default random number generator be secure?

On Mon, 14 Sep 2015 09:26:58 +0100 Robert Kern wrote: > > Actually, I meant all of the crap *around* it, the platform-compatibility > testing to see if you have such a hardware instruction or not, and C++ > template shenanigans in the surrounding code. It's possible that

Re: [Numpy-discussion] Sign of NaN

On Tue, 29 Sep 2015 09:13:15 -0600 Charles R Harris wrote: > > Due to a recent commit, Numpy master now raises an error when applying the > sign function to an object array containing NaN. Other options may be > preferable, returning NaN for instance, so I would like

Re: [Numpy-discussion] Should we drop support for "one file" compilation mode?

On Mon, 5 Oct 2015 15:26:17 -0700 Nathaniel Smith wrote: > Hi all, > > For a long time, NumPy has supported two different ways of being compiled: > > "Separate compilation" mode: like most C projects, each .c file gets > compiled to a .o file, and then the .o files get linked

Re: [Numpy-discussion] method to calculate the magnitude squared

On Fri, 18 Sep 2015 21:16:42 -0700 Phillip Feldman wrote: > In communications and signal processing, it is frequently necessary to > calculate the power of a signal. This can be done with a function like the > following: > > def magsq(z): >""" >Return the

Re: [Numpy-discussion] draft NEP for breaking ufunc ABI in a controlled way

On Thu, 24 Sep 2015 00:20:23 -0700 Nathaniel Smith wrote: > > int PyUFunc_Identity(PyFuncObject *) > > > > Replaces ufunc->identity. > > Hmm, I can imagine cases where we might want to change how this works. > (E.g. if np.dot were a ufunc then the existing identity settings >

Re: [Numpy-discussion] draft NEP for breaking ufunc ABI in a controlled way

On Mon, 21 Sep 2015 21:38:36 -0700 Nathaniel Smith <n...@pobox.com> wrote: > Hi Antoine, > > On Mon, Sep 21, 2015 at 2:44 AM, Antoine Pitrou <solip...@pitrou.net> wrote: > > > > Hi Nathaniel, > > > > On Sun, 20 Sep 2015 21:13:30 -0700 > > Nathani

Re: [Numpy-discussion] 1.10.0rc1 coming tomorrow, 22 Sept.

On Tue, 22 Sep 2015 04:43:18 -0500 Travis Oliphant wrote: > Absolutely it would be good if others can test. All I was suggesting is > that we do run a pretty decent set of tests upon build and that would be > helpful. > > If the numpy build recipes are not available, it is

Re: [Numpy-discussion] draft NEP for breaking ufunc ABI in a controlled way

Hi, This e-mail is an attempt at proposing an API to solve Numba's needs. Attribute access int PyUFunc_Nin(PyUFuncObject *) Replaces ufunc->nin. int PyUFunc_Nout(PyUFuncObject *) Replaces ufunc->nout. int PyUFunc_Nargs(PyUFuncObject *) Replaces ufunc->nargs.

Re: [Numpy-discussion] [Python-ideas] Should our default random number generator be secure?

On Mon, 14 Sep 2015 19:02:58 +0200 Sturla Molden <sturla.mol...@gmail.com> wrote: > On 14/09/15 10:34, Antoine Pitrou wrote: > > > If Numpy wanted to switch to a different generator, and if Numba wanted > > to remain compatible with Numpy, one of the PCG functions would be

Re: [Numpy-discussion] draft NEP for breaking ufunc ABI in a controlled way

Hi Nathaniel, On Sun, 20 Sep 2015 21:13:30 -0700 Nathaniel Smith wrote: > Given this, I propose that for 1.11 we: > 1) go ahead and hide/disable the problematic parts of the ABI/API, > 2) coordinate with the known affected projects to minimize disruption > to their users (which

Re: [Numpy-discussion] Should we drop support for "one file" compilation mode?

On Tue, 6 Oct 2015 09:40:43 -0700 Nathaniel Smith wrote: > > If you need some npy_* function it'd be much better to let us know > what it is and let us export it in an intentional way, instead of just > relying on whatever stuff we accidentally exposed? Ok, we seem to be using

Re: [Numpy-discussion] Should we drop support for "one file" compilation mode?

On Tue, 6 Oct 2015 10:07:13 -0700 Nathaniel Smith wrote: > > And how are you getting at them? Are you just relying the way that on > ELF systems, if two libraries are loaded into the same address space > then they automatically get access to each other's symbols, even if > they

Re: [Numpy-discussion] Numpy intermittent seg fault

Hi, On Fri, 11 Dec 2015 10:05:59 +1000 Jacopo Sabbatini wrote: > > I'm experiencing random segmentation faults from numpy. I have generated a > core dumped and extracted a stack trace, the following: > > #0 0x7f3a8d921d5d in getenv () from /lib64/libc.so.6

Re: [Numpy-discussion] Integers to integer powers, let's make a decision

On Fri, 10 Jun 2016 20:28:30 -0400 Allan Haldane wrote: > > Also, I like to think of numpy as having quite C-like behavior, allowing > you to play with the lowlevel bits and bytes. (I actually wish its > casting behavior was more C-like). I suspect that people working

Re: [Numpy-discussion] Integers to integer powers, let's make a decision

On Mon, 13 Jun 2016 10:49:44 -0400 josef.p...@gmail.com wrote: > > My argument is that `**` is like integer division and sqrt where the domain > where integer return are the correct numbers is too small to avoid > headaches by users. float64 has less integer precision than int64: >>>

Re: [Numpy-discussion] Integers to integer powers, let's make a decision

On Mon, 13 Jun 2016 10:05:08 -0400 Alan Isaac wrote: > > That is a misunderstanding, which may be influencing the discussion. > Examples of complications: > > >>> type(np.int8(2)**2) > > >>> type(np.uint64(2)**np.int8(2)) > The `uint64 x int8 -> float64` is IMHO an

Re: [Numpy-discussion] Changing FFT cache to a bounded LRU cache

On Sat, 28 May 2016 20:19:27 +0200 Sebastian Berg wrote: > > The complexity addition is a bit annoying I must admit, on python 3 > functools.lru_cache could be another option, but only there. You can backport the pure Python version of lru_cache for Python 2 (or

Re: [Numpy-discussion] Numpy 1.11.0b2 released

On Wed, 3 Feb 2016 21:56:08 -0800 Nathaniel Smith wrote: > > An extra ~2 hours of tests / 6-way parallelism is not that big a deal > in the grand scheme of things (and I guess it's probably less than > that if we can take advantage of existing binary builds) -- certainly > I can

Re: [Numpy-discussion] How to check for memory leaks?

On Tue, 23 Feb 2016 12:36:00 -0700 Charles R Harris wrote: > Hi All, > > I'm suspecting a possible memory leak in 1.11.x, what is the best way to > check for that? If that is due to a reference leak, you can use sys.getrefcount() or weakref.ref(). Otherwise you may

Re: [Numpy-discussion] Multidimension array access in C via Python API

On Tue, 5 Apr 2016 08:39:39 -0700 (MST) mpc wrote: > This is the reason I'm doing this in the first place, because I made a pure > python version but it runs really slow for larger data sets, so I'm > basically rewriting the same function but using the Python and Numpy C

Re: [Numpy-discussion] tracemalloc + numpy?

On Tue, 08 Mar 2016 09:26:13 -0500 Neal Becker wrote: > I'm trying tracemalloc to find memory usage. Will numpy array memory usage > be counted by tracemalloc? (Doesn't seem to) No, but together with something like https://github.com/numpy/numpy/pull/5470 it could.

Re: [Numpy-discussion] Floor divison on int returns float

On Wed, 13 Apr 2016 15:49:15 -0600 Charles R Harris wrote: > > I looked this up once, `C` returns unsigned in the scalar case when both > operands have the same width. See Usual Arithmetic Conversions >

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On Thu, 12 May 2016 23:14:36 + (UTC) Sturla Molden <sturla.mol...@gmail.com> wrote: > Antoine Pitrou <solip...@pitrou.net> wrote: > > > Can you define "expensive"? > > Slow enough to cause complaints on the Cython mailing list. What kind of com

Re: [Numpy-discussion] Integers to integer powers

On Thu, 19 May 2016 20:30:40 -0700 Nathaniel Smith wrote: > > Given these immutable and contradictory constraints, the last bad > option IMHO would be that we make int ** (negative int) an error in > all cases, and the error message can suggest that instead of writing > >

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

On Thu, 12 May 2016 06:27:43 + (UTC) Sturla Molden wrote: > Allan Haldane wrote: > > > You probably already know this, but I just wanted to note that the > > mpi4py module has worked around pickle too. They discuss how they > > efficiently

Re: [Numpy-discussion] Calling C code that assumes SIMD aligned data.

Here's an obligatory plug for the two following PRs: https://github.com/numpy/numpy/pull/5457 https://github.com/numpy/numpy/pull/5470 Regards Antoine. On Fri, 6 May 2016 15:01:32 +0200 Francesc Alted wrote: > 2016-05-05 22:10 GMT+02:00 Øystein Schønning-Johansen

Re: [Numpy-discussion] State-of-the-art to use a C/C++ library from Python

On Fri, 2 Sep 2016 02:16:35 -0700 Nathaniel Smith wrote: > > > > Depending on how minimal and universal you want to keep things, I use > > the ctypes approach quite often, i.e. treat your numpy inputs an > > outputs as arrays of doubles etc using the ndpointer(...) syntax. I > >

Re: [Numpy-discussion] Default type for functions that accumulate integers

On Mon, 2 Jan 2017 18:46:08 -0800 Nathaniel Smith wrote: > > So some options include: > - make the default integer precision 64-bits everywhere > - make the default integer precision 32-bits on 32-bit systems, and > 64-bits on 64-bit systems (including Windows) Either of those