[Numpy-discussion] Re: Fastpathing lexsort for integers

call using private APIs like _multiarray_umath. Obviously writing CPython C API code is a lot less straightforward than Python, but the numpy code reviewers have a lot of experience spotting C API issues and we can point you to resources for learning. Hope that helps, Nathan On Sat, May 11, 2024

[Numpy-discussion] Re: Add context management for np.seterr

I think you're looking for the errstate context manager: https://numpy.org/doc/stable/reference/generated/numpy.errstate.html On Thu, May 9, 2024 at 1:11 PM wrote: > The current way (according to 1.26 doc) of setting and resetting error is > ``` > old_settings = np.seterr(all='ignore')

[Numpy-discussion] Re: PR - can I get a new review?

I think most of the build failures you’re seeing would be fixed by merging with or rebasing on the latest main branch. Note that there is currently an issue with some of the windows CI runners, so you’ll see failures related to our spin configuration failing to handle a gcov argument that was

[Numpy-discussion] Re: Moving the weekly traige/community meetings

That time work for me, I have a conflict with the old time an hour earlier than the current time so hopefully that works for everyone. On Sun, Apr 7, 2024 at 8:34 PM Matti Picus wrote: > Could we move the weekly community/triage meetings one hour later? Some > participants have a permanent

[Numpy-discussion] Re: dtype=object arrays not treated like Python list?

On Fri, Mar 29, 2024 at 8:39 AM Jim Pivarski wrote: > On Fri, Mar 29, 2024 at 8:07 AM Steven G. Johnson wrote: > >> Should a dtype=object array be treated more like Python lists for type >> detection/coercion reasons? Currently, they are treated quite differently: >> >>> np.isfinite([1,2,3])

[Numpy-discussion] Re: Arrays of variable itemsize

that all elements of an array are still of the > same size in memory? > > I am thinking along the lines of per-element dynamic memory management. > Such that if I had array [0, 1e1], the first element would default to > reasonably small size in memory. > > On 13 Mar 2024, at 16:29,

[Numpy-discussion] Re: Arrays of variable itemsize

It is possible to do this using the new DType system. Sebastian wrote a sketch for a DType backed by the GNU multiprecision float library: https://github.com/numpy/numpy-user-dtypes/tree/main/mpfdtype It adds a significant amount of complexity to store data outside the array buffer and

[Numpy-discussion] Re: ENH: Add 'axis' kwarg to np.tile

keywords in the API and now that the array API is a thing, we can point to that as a place to hash out API decisions. Including syntax in the array API also encourages more libraries to adopt your preferred syntax. Nathan On Thu, Feb 29, 2024 at 4:12 PM wrote: > Hoping to get some more feedb

[Numpy-discussion] Re: Resources of newcomers and some questions about my first PR

On Mon, Feb 19, 2024 at 3:20 PM Amogh Sood wrote: > Hi all. > > I want to thank you all for your contributions and efforts in maintaining > numpy. > numpy has been an indispensable tool during both my graduate and > undergraduate studies. > > While I am no software wizard, going forward, I do

[Numpy-discussion] New DType and ArrayMethod C APIs are public

Walt, Ben Nathanson, Marten van Kerkwijk, who co-authored the NEPs, Ralf Gommers who helped get funding for my work and provided mentoring and coordination, Charles Harris for providing leadership, context, and advice, and many others who have contributed in big and small ways

[Numpy-discussion] Re: NEP 55 - Add a UTF-8 Variable-Width String DType to NumPy

ay entry without adding complexity or needing to re-allocate the entire arena buffer. For example, if you create an array with a short string entry and then edit that entry to be longer than 15 bytes. Rather than appending to the arena or re-allocating it, we convert the entry to a heap string. > Thank

[Numpy-discussion] Proposal to accept NEP 55: Add a UTF-8 variable-width string DType to NumPy

Hi all, I propose we accept NEP 55 and merge PR #25347 implementing the NEP in time for the NumPy 2.0 RC: https://numpy.org/neps/nep-0055-string_dtype.html https://github.com/numpy/numpy/pull/25347 The most controversial aspect of the NEP was support for missing strings via a user-supplied

[Numpy-discussion] Re: NEP 55 Updates and call for testing

here, on github, or reach out to me privately. On Wed, Nov 22, 2023 at 1:22 PM Nathan wrote: > Hi all, > > This week I updated NEP 55 to reflect the changes I made to the prototype > since > I initially sent out the NEP. The updated NEP is available on the NumPy > website: > h

[Numpy-discussion] Re: Meson - C extension - Finding numpy includes in virtual env

Nov 26, 2023, at 12:03, Nathan wrote: > > For my work I tend to use a persistent build directory with build > isolation disabled as discussed in the meson-python docs. > > > Out of curiosity, how is this different from, e.g., `spin build` which > builds into `./bu

[Numpy-discussion] Re: Meson - C extension - Finding numpy includes in virtual env

I want to caution about using `pip install -e .` to get a development install of numpy. This will work fine working on numpy itself, but won’t be useful if you need to use the development version of numpy to build another library. This doesn’t work because in-place installs don’t install the numpy

[Numpy-discussion] NEP 55 Updates and call for testing

, attention, and help reviewing the NEP! -Nathan ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address

[Numpy-discussion] Re: Adding NumpyUnpickler to Numpy 1.26 and future Numpy 2.0

e and use attribute access or use `from` imports, which both invoke the module-level `__getattr__`. Hopefully we’ll get most of the benefit of alerting users that they are using private internals without needing to break old pickles. > > > On Tue, Oct 10, 2023 at 3:33 PM Nathan

[Numpy-discussion] Re: Adding NumpyUnpickler to Numpy 1.26 and future Numpy 2.0

On Tue, Oct 10, 2023 at 7:03 AM Ronald van Elburg < r.a.j.van.elb...@hetnet.nl> wrote: > I have one more useCase to consider from our ecosystem. > > We dump numpy arrays into a MongoDB using GridFS for subsequent > visualization, some snippets: > > '''Python > with BytesIO() as BIO: >

[Numpy-discussion] Re: Adding NumpyUnpickler to Numpy 1.26 and future Numpy 2.0

On Mon, Oct 9, 2023 at 3:58 PM Oscar Benjamin wrote: > On Mon, 9 Oct 2023 at 22:30, Nathan wrote: > > > > On Mon, Oct 9, 2023 at 3:12 PM Oscar Benjamin < > oscar.j.benja...@gmail.com> wrote: > >> > >> On Mon, 9 Oct 2023 at 21:57, Nathan wrote: > >

[Numpy-discussion] Re: Adding NumpyUnpickler to Numpy 1.26 and future Numpy 2.0

On Mon, Oct 9, 2023 at 3:12 PM Oscar Benjamin wrote: > On Mon, 9 Oct 2023 at 21:57, Nathan wrote: > > > > On Mon, Oct 9, 2023 at 2:44 PM Oscar Benjamin < > oscar.j.benja...@gmail.com> wrote: > >> Suppose that there is NumPy v1 and that in future there wi

[Numpy-discussion] Re: Adding NumpyUnpickler to Numpy 1.26 and future Numpy 2.0

On Mon, Oct 9, 2023 at 2:44 PM Oscar Benjamin wrote: > On Mon, 9 Oct 2023 at 17:03, Nathan wrote: > > > > On Mon, Oct 9, 2023 at 12:57 AM Aaron Meurer wrote: > >> > >> Is it possible to convert a NumPy 1 pickle file into a generic pickle > >> file tha

[Numpy-discussion] Re: Adding NumpyUnpickler to Numpy 1.26 and future Numpy 2.0

people to continue loading old pickle files indefinitely (albeit with a minor code change). This also gives us a place to put compatibility fixes for future changes that impact old pickle files. -Nathan > > Aaron Meurer > > On Fri, Oct 6, 2023 at 10:17 AM Nathan wrote: > > > &

[Numpy-discussion] Re: Adding NumpyUnpickler to Numpy 1.26 and future Numpy 2.0

I don’t think this will be a problem for using pickle for IPC. For the python multiprocessing module, all processes would be running the same numpy version, so there wouldn’t be a problem. It could be an issue if pickle is used to communicate numpy arrays between a subset of workers running

[Numpy-discussion] Adding NumpyUnpickler to Numpy 1.26 and future Numpy 2.0

/numpy/issues/24844, and the discussion in https://github.com/numpy/numpy/pull/24634 for more context. -Nathan ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https

[Numpy-discussion] Re: NEP 55 - Add a UTF-8 Variable-Width String DType to NumPy

gt; >>> >>> On Fri, Sep 15, 2023 at 3:18 PM Warren Weckesser < >>> warren.weckes...@gmail.com> wrote: >>> > >>> > >>> > >>> > On Mon, Sep 11, 2023 at 12:25 PM Nathan >>> wrote: >>> >> >>> >

[Numpy-discussion] Re: NEP 55 - Add a UTF-8 Variable-Width String DType to NumPy

On Wed, Sep 20, 2023 at 12:26 AM Warren Weckesser < warren.weckes...@gmail.com> wrote: > > > On Fri, Sep 15, 2023 at 3:18 PM Warren Weckesser < > warren.weckes...@gmail.com> wrote: > > > > > > > > On Mon, Sep 11, 2023 at 12:25 PM Nathan > wrot

[Numpy-discussion] Re: Curious performance different with np.unique on arrays of characters

Looking at a py-spy profile of a slightly modified version of the code you shared, it seems the difference comes down to NumPy's sorting implementation simply being faster for ints than unicode strings. In particular, it looks like string_quicksort_ is two or three times slower than quicksort_

[Numpy-discussion] Re: NEP 55 - Add a UTF-8 Variable-Width String DType to NumPy

On Sun, Sep 3, 2023 at 10:54 AM Warren Weckesser wrote: > > > On Tue, Aug 29, 2023 at 10:09 AM Nathan wrote: > > > > The NEP was merged in draft form, see below. > > > > https://numpy.org/neps/nep-0055-string_dtype.html > > > > On Mon, Aug 21, 202

[Numpy-discussion] Re: NEP 55 - Add a UTF-8 Variable-Width String DType to NumPy

The NEP was merged in draft form, see below. https://numpy.org/neps/nep-0055-string_dtype.html On Mon, Aug 21, 2023 at 2:36 PM Nathan wrote: > Hello all, > > I just opened a pull request to add NEP 55, see > https://github.com/numpy/numpy/pull/24483. > > Per NEP 0, I've co

[Numpy-discussion] NEP 55 - Add a UTF-8 Variable-Width String DType to NumPy

Hello all, I just opened a pull request to add NEP 55, see https://github.com/numpy/numpy/pull/24483. Per NEP 0, I've copied everything up to the "detailed description" section below. I'm looking forward to your feedback on this. -Natha

[Numpy-discussion] Re: Add to NumPy a function to compute cumulative sums from 0.

This has come up before, see https://github.com/numpy/numpy/issues/6044 for the first time this came up; there were several subsequent discussions linked there. In the meantime, the data APIs consortium has been actively working on adding a `cumulative_sum` function to the array API standard, see

[Numpy-discussion] New user dtypes and the buffer protocol

Hi all, As you may know, I'm currently working on a variable-width string dtype using the new experimental user dtype API. As part of this work I'm running into papercuts that future dtype authors will likely hit and I've been trying to fix them as I go. One issue I'd like to raise with the list

[Numpy-discussion] Re: Welcome Nathan Goldbaum as a Maintainer

Thanks so much everyone! On Mon, May 29, 2023 at 9:00 AM Ross Barnowski wrote: > Welcome Nathan! > > On Mon, May 29, 2023 at 7:47 AM Charles R Harris < > charlesr.har...@gmail.com> wrote: > >> >> >> On Mon, May 29, 2023 at 1:15 AM Sebastian Berg < >&

[Numpy-discussion] Re: Introducing `np.types`

On Mon, Feb 13, 2023 at 4:34 AM Sebastian Berg wrote: > On Sat, 2023-02-11 at 11:24 +, Ralf Gommers wrote: > > On Fri, Feb 10, 2023 at 5:35 PM Nathan > > wrote: > > > > > > > > > > > > > > Small bikeshed: the name np.types indicates

[Numpy-discussion] Re: Introducing `np.types`

On Fri, Feb 10, 2023 at 3:31 AM Sebastian Berg wrote: > Hi all, > > I was wondering if we should introduce a new `np.types` namespace. The > main reason is that we have the DType classes, that most users don't > need to worry about. These mirror the scalar classes, but getting them > is weird

[Numpy-discussion] New API for testing custom array containers

the discussion in https://github.com/numpy/numpy/issues/15544 for more context. -Nathan ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists

Re: [Numpy-discussion] Py-API: Deprecate `np.dtype(np.floating)` and similar dtype creation

Yeah, that seems to be more popular: https://github.com/search?q=%22dtype%3Dnp.integer%22=Code On Fri, Feb 14, 2020 at 2:45 PM Sebastian Berg wrote: > On Fri, 2020-02-14 at 14:39 -0700, Nathan wrote: > > For what it's worth, github search only finds two instances of this

Re: [Numpy-discussion] Py-API: Deprecate `np.dtype(np.floating)` and similar dtype creation

For what it's worth, github search only finds two instances of this usage: https://github.com/search?q=%22np.dtype%28np.floating%29%22=Code On Fri, Feb 14, 2020 at 2:28 PM Sebastian Berg wrote: > Hi all, > > In https://github.com/numpy/numpy/pull/15534 I would like to start > deprecating

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 7:27 PM Nathaniel Smith wrote: > On Sun, Sep 8, 2019 at 8:40 AM Ralf Gommers > wrote: > > > > > > > > On Sun, Sep 8, 2019 at 12:54 AM Nathaniel Smith wrote: > >> > >> On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers > wrote: > >> > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel

Re: [Numpy-discussion] Converting np.sinc into a ufunc

It might be worth using BigQuery to search the github repository public dataset for usages of np.sinc with keyword arguments. On Wed, May 22, 2019 at 1:05 PM Sebastian Berg wrote: > Hi all, > > there is an open PR (https://github.com/numpy/numpy/pull/12924) to > convert `np.sinc` into a ufunc.

[Numpy-discussion] [ANN] 2019 Scipy Conference: Call for Proposals

SciPy 2019, the 18th annual Scientific Computing with Python conference, will be held July 8-14, 2019 in Austin, Texas. The annual SciPy Conference brings together over 800 participants from industry, academia, and government to showcase their latest projects, learn from skilled users and

Re: [Numpy-discussion] Taking back control of the #numpy irc channel

of traffic (perhaps 10% of the number of questions that get asked on StackOverflow) and it’s a good venue for asking quick questions. Let’s hope that forcing people to register doesn’t kill that, although there’s not much we can do given the spam attack. Nathan On Mon, Aug 6, 2018 at 9:03 PM Matti Picus

Re: [Numpy-discussion] Adoption of a Code of Conduct

On Wed, Aug 1, 2018 at 9:49 AM, Ralf Gommers wrote: > > > On Wed, Aug 1, 2018 at 12:20 AM, Nathan Goldbaum > wrote: > >> I realize this was probably brought up in the discussions about the scipy >> code of conduct which I have not looked at, but I’m troubled by the

Re: [Numpy-discussion] NumPy 1.15.0rc2 released.

Hi Chuck, Is there a summary of the differences with respect to rc1 somewhere? Nathan On Mon, Jul 9, 2018 at 5:08 PM Charles R Harris wrote: > Hi All, > > On behalf of the NumPy team I'm pleased to announce the release of NumPy > 1.15.0rc2. > This release has an unusual num

Re: [Numpy-discussion] PEP 574 - zero-copy pickling with out of band data

On Mon, Jul 2, 2018 at 7:42 PM Andrew Nelson wrote: > > > On Tue, 3 Jul 2018 at 09:31, Charles R Harris > wrote: > >> >> ISTR that some parallel processing applications sent pickled arrays >> around to different processes, I don't know if that is still the case, but >> if so, no copy might be

Re: [Numpy-discussion] rackspace ssl certificates

I think Matthew Brett needs to fix this. On Mon, Jun 18, 2018 at 3:20 PM Charles R Harris wrote: > Hi All, > > I've been trying to put out the NumPy 1.15.0rc1, but cannot get > `numpy-wheels` to upload the wheels to rackspace on windows, there is a > certification problem. I note that that

Re: [Numpy-discussion] Updated 1.15.0 release notes

with the prerelease so numpy and other projects can implement workarounds as needed without doing a release that might potentially break real users who happen to install right after numpy 1.x.0 comes out. Best, Nathan Goldbaum On Wed, Jun 13, 2018 at 7:11 PM Charles R Harris wrote: > Hi

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Hmm, does this mean the callable that gets passed into __array_ufunc__ will change? I'm pretty sure that will break the dispatch mechanism I'm using in my __array_ufunc__ implementation, which directly checks whether the callable is in one of several tuples of functions that have different

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

Perhaps I missed this but I didn’t see: what happens when both __array_ufunc__ and __array_function__ are defined? I might want to do this to for example add support for functions like concatenate or stack to a class that already has an __array_ufunc__ defines. On Sat, Jun 2, 2018 at 5:56 PM

[Numpy-discussion] Citation for ndarray

tion exist? Thanks! -Nathan ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Turn numpy.ones_like into a ufunc

I don't particularly need this, although it would be nice to make this behavior explicit, instead of happening more or less by accident: In [1]: from yt.units import km In [2]: import numpy as np In [3]: data = [1, 2, 3]*km In [4]: np.ones_like(data) Out[4]: YTArray([1.,

Re: [Numpy-discussion] Casting scalars

On Thu, May 10, 2018 at 9:51 PM Stuart Reynolds wrote: > np.float(scalar) > This actually isn't right. It's a common misconception, but np.float is an alias to the built-in float type. You probably want np.float_(scalar) In [5]: np.float_(12).dtype Out[5]:

Re: [Numpy-discussion] Casting scalars

In [1]: import numpy as np In [2]: np.float64(12) Out[2]: 12.0 In [3]: np.float64(12).dtype Out[3]: dtype('float64') On Thu, May 10, 2018 at 9:49 PM Hameer Abbasi wrote: > Hello, everyone! > > I might be missing something and this might be a very stupid and

Re: [Numpy-discussion] Short-circuiting equivalent of np.any or np.all?

least that way others won't need to deal with the pain :) > - Joe > > On Thu, Apr 26, 2018 at 12:51 PM, Hameer Abbasi <einstein.edi...@gmail.com > > wrote: > >> Hi Nathan, >> >> np.any and np.all call np.or.reduce and np.and.reduce respectively, and >> unf

Re: [Numpy-discussion] Short-circuiting equivalent of np.any or np.all?

On Thu, Apr 26, 2018 at 11:52 AM Hameer Abbasi <einstein.edi...@gmail.com> wrote: > Hi Nathan, > > np.any and np.all call np.or.reduce and np.and.reduce respectively, and > unfortunately the underlying function (ufunc.reduce) has no way of > detecting that the value isn’t go

[Numpy-discussion] Short-circuiting equivalent of np.any or np.all?

and conda-forge packages set up on all platforms. Thanks for your help! -Nathan ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Introduction: NumPy developers at BIDS

On Tue, Apr 10, 2018 at 9:59 AM, Stefan van der Walt wrote: > Hi Eric, > > On Sun, 08 Apr 2018 08:02:19 -1000, Eric Firing wrote: > > On 2018/04/07 9:19 PM, Stefan van der Walt wrote: > > > We would love community input on identifying the best areas & issues to > > > pay

Re: [Numpy-discussion] is __array_ufunc__ ready for prime-time?

On Thu, Nov 2, 2017 at 5:21 PM, Stephan Hoyer <sho...@gmail.com> wrote: > On Thu, Nov 2, 2017 at 12:42 PM Nathan Goldbaum <nathan12...@gmail.com> > wrote: > >> Would this issue be ameliorated given Nathaniel's proposal to try to move >> away from subclasses a

Re: [Numpy-discussion] is __array_ufunc__ ready for prime-time?

I think the biggest issues could be resolved if __array_concatenate__ were finished. Unfortunately I don't feel like I can take that on right now. See Ryan May's talk at scipy about using an ndarray subclass for units and the issues he's run into: https://www.youtube.com/watch?v=qCo9bkT9sow On

Re: [Numpy-discussion] is __array_ufunc__ ready for prime-time?

I’m using it in yt. If we were able to drop support for all old numpy versions, switching would allow me to delete hundreds of lines of code. As-is since we need to simultaneously support old and new versions, it adds some additional complexity. If you’re ok with only supporting numpy >= 1.13,

Re: [Numpy-discussion] numpy grant update

these days. Please correct me if I'm wrong, Nathaniel. -Nathan On Thu, Oct 26, 2017 at 5:14 PM, Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Hi Nathaniel, > > Thanks for the link. The plans sounds great! You'll not be surprised > to hear I'm particularly inter

Re: [Numpy-discussion] Github overview change

This is a change in the UI that github introduced a couple weeks ago during their annual conference. See https://github.com/blog/2447-a-more-connected-universe On Wed, Oct 18, 2017 at 11:49 AM Charles R Harris wrote: > On Wed, Oct 18, 2017 at 7:23 AM, Sebastian Berg

Re: [Numpy-discussion] Dropping support for Accelerate

See https://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html and replies in that thread. Quote from an Apple engineer in that thread: "For API outside of POSIX, including GCD and technologies like Accelerate, we do not support usage on both sides of a fork(). For this reason

Re: [Numpy-discussion] Controlling NumPy __mul__ method or forcing it to use __rmul__ of the "other"

I don't think there's any real standard here. Just doing a github search reveals many different choices people have used: https://github.com/search?l=Python=__array_priority__=Code=%E2%9C%93 On Mon, Jun 19, 2017 at 11:07 AM, Ilhan Polat wrote: > Thank you. I didn't know

Re: [Numpy-discussion] __array_ufunc__ counting down to launch, T-24 hrs.

Thanks for linking to the updated NEP, I've been looking for a good overview of this discussion. Up until now I haven't wanted to wade through the extensive discussion on this topic. I'm curious, if I want to simultaneously support older Numpy versions as well as newer versions, will I be able to