Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Wed, Oct 9, 2019 at 6:00 PM Juan Nunez-Iglesias wrote: > Hi all, and thank you for all your hard work with this. > > I wanted to provide more of an "end user" perspective than I think has > been present in this discussion so far. Over the past month, I've quickly > skimmed some emails on this

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

Hi all, and thank you for all your hard work with this. I wanted to provide more of an "end user" perspective than I think has been present in this discussion so far. Over the past month, I've quickly skimmed some emails on this thread and skipped others altogether. I am far from a NumPy

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Wed, Sep 11, 2019 at 4:18 PM Ralf Gommers wrote: > > > On Tue, Sep 10, 2019 at 10:59 AM Stephan Hoyer wrote: > >> On Tue, Sep 10, 2019 at 6:06 AM Hameer Abbasi >> wrote: >> >>> On 10.09.19 05:32, Stephan Hoyer wrote: >>> >>> On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers >>> wrote: >>> I

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Tue, Sep 10, 2019 at 10:59 AM Stephan Hoyer wrote: > On Tue, Sep 10, 2019 at 6:06 AM Hameer Abbasi > wrote: > >> On 10.09.19 05:32, Stephan Hoyer wrote: >> >> On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers >> wrote: >> >>> I think we've chosen to try the former - dispatch on functions so we

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Tue, Sep 10, 2019 at 10:53 AM Sebastian Berg wrote: > On Tue, 2019-09-10 at 17:28 +0200, Hameer Abbasi wrote: > > On 07.09.19 22:06, Sebastian Berg wrote: > > > > > > Now for the end-users choosing one array-like over another, seems > > > nicer > > > as an implicit mechanism (why should I not

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Tue, Sep 10, 2019 at 6:06 AM Hameer Abbasi wrote: > On 10.09.19 05:32, Stephan Hoyer wrote: > > On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers > wrote: > >> I think we've chosen to try the former - dispatch on functions so we can >> reuse the NumPy API. It could work out well, it could give

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Tue, 2019-09-10 at 17:28 +0200, Hameer Abbasi wrote: > On 07.09.19 22:06, Sebastian Berg wrote: > > On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: > > > > > > > > Let me try to move the discussion from the github issue here (this > > may > > not be the best place). ( > >

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 07.09.19 22:06, Sebastian Berg wrote: On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: Let me try to move the discussion from the github issue here (this may not be the best place). (https://github.com/numpy/numpy/issues/14441 which asked for easier creation functions together with

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 10.09.19 05:32, Stephan Hoyer wrote: On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers > wrote: I think we've chosen to try the former - dispatch on functions so we can reuse the NumPy API. It could work out well, it could give some long-term maintenance

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 09.09.19 03:26, Nathaniel Smith wrote: [snip] Generic in the sense that you can write __array_ufunc__ once and have it work for all ufuncs. You can do that too with __ua_function__, you get np.ufunc.__call__, with self=. The same holds for say, RandomState objects, once implemented. Most

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 08.09.19 10:56, Nathaniel Smith wrote: On Sun, Sep 8, 2019 at 1:04 AM Hameer Abbasi wrote: On 08.09.19 09:53, Nathaniel Smith wrote: OTOH, __array_function__ doesn't allow this kind of simplification: if we were using __array_function__ for ufuncs, every library would have to special-case

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, 2019-09-09 at 22:26 -0700, Eric Wieser wrote: > > In other words `np.arange(100)` (but > with a completely different syntax, probably hidden away only for > libraries to use). > > It sounds an bit like you're describing factory classmethods there. > Is the solution to this problem to move

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

> In other words `np.arange(100)` (but with a completely different syntax, probably hidden away only for libraries to use). It sounds an bit like you're describing factory classmethods there. Is the solution to this problem to move (leaving behind aliases) `np.arange` to `ndarray.arange`,

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, 2019-09-09 at 20:32 -0700, Stephan Hoyer wrote: > On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers > wrote: > > I think we've chosen to try the former - dispatch on functions so > > we can reuse the NumPy API. It could work out well, it could give > > some long-term maintenance issues, time

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, Sep 9, 2019 at 6:27 PM Ralf Gommers wrote: > I think we've chosen to try the former - dispatch on functions so we can > reuse the NumPy API. It could work out well, it could give some long-term > maintenance issues, time will tell. The question is now if and how to plug > the gap that

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 6:27 PM Nathaniel Smith wrote: > On Sun, Sep 8, 2019 at 8:40 AM Ralf Gommers > wrote: > > > > > > > > On Sun, Sep 8, 2019 at 12:54 AM Nathaniel Smith wrote: > >> > >> On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers > wrote: > >> > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 7:27 PM Nathaniel Smith wrote: > On Sun, Sep 8, 2019 at 8:40 AM Ralf Gommers > wrote: > > > > > > > > On Sun, Sep 8, 2019 at 12:54 AM Nathaniel Smith wrote: > >> > >> On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers > wrote: > >> > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 8:40 AM Ralf Gommers wrote: > > > > On Sun, Sep 8, 2019 at 12:54 AM Nathaniel Smith wrote: >> >> On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers wrote: >> > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel Smith wrote: >> >> On Tue, Sep 3, 2019 at 2:04 AM Hameer Abbasi >> >>

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 12:54 AM Nathaniel Smith wrote: > On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers > wrote: > > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel Smith wrote: > >> On Tue, Sep 3, 2019 at 2:04 AM Hameer Abbasi > wrote: > >> > The fact that we're having to design more and more

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 08.09.19 10:56, Nathaniel Smith wrote: On Sun, Sep 8, 2019 at 1:04 AM Hameer Abbasi wrote: On 08.09.19 09:53, Nathaniel Smith wrote: OTOH, __array_function__ doesn't allow this kind of simplification: if we were using __array_function__ for ufuncs, every library would have to special-case

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sun, Sep 8, 2019 at 1:04 AM Hameer Abbasi wrote: > > On 08.09.19 09:53, Nathaniel Smith wrote: >> OTOH, __array_function__ doesn't allow this kind of simplification: if >> we were using __array_function__ for ufuncs, every library would have >> to special-case every individual ufunc, which

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 08.09.19 09:53, Nathaniel Smith wrote: On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers wrote: On Fri, Sep 6, 2019 at 12:53 AM Nathaniel Smith wrote: On Tue, Sep 3, 2019 at 2:04 AM Hameer Abbasi wrote: The fact that we're having to design more and more protocols for a lot of very similar

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 11:53 AM Ralf Gommers wrote: > On Fri, Sep 6, 2019 at 12:53 AM Nathaniel Smith wrote: >> On Tue, Sep 3, 2019 at 2:04 AM Hameer Abbasi >> wrote: >> > The fact that we're having to design more and more protocols for a lot >> > of very similar things is, to me, an indicator

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sat, Sep 7, 2019 at 5:08 PM Ralf Gommers wrote: > > On Sat, Sep 7, 2019 at 4:16 PM Nathaniel Smith wrote: >> >> On Fri, Sep 6, 2019 at 11:04 PM Ralf Gommers wrote: >> > Vendoring means "include the code". So no dependency on an external >> > package. If we don't vendor, it's going to be

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sat, Sep 7, 2019 at 4:16 PM Nathaniel Smith wrote: > On Fri, Sep 6, 2019 at 11:04 PM Ralf Gommers > wrote: > > Vendoring means "include the code". So no dependency on an external > package. If we don't vendor, it's going to be either unused, or end up as a > dependency for the whole

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 11:04 PM Ralf Gommers wrote: > Vendoring means "include the code". So no dependency on an external package. > If we don't vendor, it's going to be either unused, or end up as a dependency > for the whole SciPy/PyData stack. If we vendor it then it also ends up as a

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sat, Sep 7, 2019 at 2:18 PM sebastian wrote: > On 2019-09-07 15:33, Ralf Gommers wrote: > > On Sat, Sep 7, 2019 at 1:07 PM Sebastian Berg > > wrote: > > > >> On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: > >>> > >>> > >> > >> > That's part of it. The concrete problems it's

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On 2019-09-07 15:33, Ralf Gommers wrote: On Sat, Sep 7, 2019 at 1:07 PM Sebastian Berg wrote: On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: That's part of it. The concrete problems it's solving are threefold: Array creation functions can be overridden. Array coercion is now

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Sat, Sep 7, 2019 at 1:07 PM Sebastian Berg wrote: > On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: > > > > > > > > > That's part of it. The concrete problems it's solving are > > > threefold: > > > Array creation functions can be overridden. > > > Array coercion is now covered. > > >

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, 2019-09-06 at 14:45 -0700, Ralf Gommers wrote: > > > > That's part of it. The concrete problems it's solving are > > threefold: > > Array creation functions can be overridden. > > Array coercion is now covered. > > "Default implementations" will allow you to re-write your NumPy > >

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

>> There may be another very concrete one (that's not yet in the NEP): allowing >> other libraries that consume ndarrays to use overrides. An example is >> numpy.fft: currently both mkl_fft and pyfftw monkeypatch NumPy, something we >> don't like all that much (in particular for mkl_fft,

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 4:51 PM Nathaniel Smith wrote: > On Fri, Sep 6, 2019 at 2:45 PM Ralf Gommers > wrote: > > There may be another very concrete one (that's not yet in the NEP): > allowing other libraries that consume ndarrays to use overrides. An example > is numpy.fft: currently both

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 5:16 PM Nathaniel Smith wrote: > On Fri, Sep 6, 2019 at 11:44 AM Ralf Gommers > wrote: > > > > > > > > On Fri, Sep 6, 2019 at 1:32 AM Hameer Abbasi > wrote: > >> > >> That's a lot of very good questions! Let me see if I can answer them > one-by-one. > >> > >> On 06.09.19

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 2:45 PM Ralf Gommers wrote: > There may be another very concrete one (that's not yet in the NEP): allowing > other libraries that consume ndarrays to use overrides. An example is > numpy.fft: currently both mkl_fft and pyfftw monkeypatch NumPy, something we > don't like

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 1:32 AM Hameer Abbasi wrote: > That's a lot of very good questions! Let me see if I can answer them > one-by-one. > > On 06.09.19 09:49, Nathaniel Smith wrote: > > But those are general questions about unumpy, and I'm guessing no-one > knows all the answers yet... and

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 12:53 AM Nathaniel Smith wrote: > On Mon, Sep 2, 2019 at 11:21 PM Ralf Gommers > wrote: > > On Mon, Sep 2, 2019 at 2:09 PM Nathaniel Smith wrote: > > On Tue, Sep 3, 2019 at 2:04 AM Hameer Abbasi > wrote: > > The fact that we're having to design more and more protocols

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Fri, Sep 6, 2019 at 1:32 AM Hameer Abbasi wrote: > That's a lot of very good questions! Let me see if I can answer them > one-by-one. > > On 06.09.19 09:49, Nathaniel Smith wrote: > > But even that could be accomplished by just > putting something in the docs. And adding the alias has

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

That's a lot of very good questions! Let me see if I can answer them one-by-one. On 06.09.19 09:49, Nathaniel Smith wrote: Ah, whoops, I definitely missed that :-). That does change things! So one of the major decision points for any duck-array API work, is whether to modify the numpy

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

Hello everyone; Thanks to all the feedback from the community, in particular Sebastian Berg, we have a new draft of NEP-31. Please find the full text quoted below for discussion and reference. Any feedback and discussion is welcome.

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

Hi Nathaniel, On 02.09.19 23:09, Nathaniel Smith wrote: On Mon, Sep 2, 2019 at 2:15 AM Hameer Abbasi wrote: Me, Ralf Gommers and Peter Bell (both cc’d) have come up with a proposal on how to solve the array creation and duck array problems. The solution is outlined in NEP-31, currently in

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, Sep 2, 2019 at 2:09 PM Nathaniel Smith wrote: > On Mon, Sep 2, 2019 at 2:15 AM Hameer Abbasi > wrote: > > Me, Ralf Gommers and Peter Bell (both cc’d) have come up with a proposal > on how to solve the array creation and duck array problems. The solution is > outlined in NEP-31,

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

On Mon, Sep 2, 2019 at 2:15 AM Hameer Abbasi wrote: > Me, Ralf Gommers and Peter Bell (both cc’d) have come up with a proposal on > how to solve the array creation and duck array problems. The solution is > outlined in NEP-31, currently in the form of a PR, [1] Thanks for putting this

Re: [Numpy-discussion] NEP 31: Context-local and global overrides of the NumPy API

The full-text of the NEP is quoted below, apologies for not including it earlier: NEP 31 — Context-local and global overrides of the NumPy API :Author: Hameer Abbasi