Re: [Numpy-discussion] beginner introduction to group
On Sat, Sat, 25 Apr 2020 01:59:57 Robert Kern wrote: Welcome! Do you have an idea what you would like to work on? Hi Robert, Nothing specific for now, But I am at present trying to work on Issue #15961. Titled "Einsum indexing very fragile, because it tests for int(and int64 is not int). Tina Oberoi ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add `force=` or `copy=` kwarg to `__array__` interface
On Sat, Apr 25, 2020 at 10:40 AM Ralf Gommers wrote: > > > On Fri, Apr 24, 2020 at 12:35 PM Eric Wieser > wrote: > >> Perhaps worth mentioning that we've discussed this sort of API before, in >> https://github.com/numpy/numpy/pull/11897. >> >> Under that proposal, the api would be something like: >> >> * `copy=True` - always copy, like it is today >> * `copy=False` - copy if needed, like it is today >> * `copy=np.never_copy` - never copy, throw an exception if not possible >> > > There's a couple of issues I see with using `copy` for __array__: > - copy is already weird (False doesn't mean no), and a [bool, > some_obj_or_str] keyword isn't making that better > - the behavior we're talking about can do more than copying, e.g. for > PyTorch it would modify the autograd graph by adding detach(), and for > sparse it's not just "make a copy" (which implies doubling memory use) but > it densifies which can massively blow up the memory. > - I'm -1 on adding things to the main namespace (never_copy) for something > that can be handled differently (like a string, or a new keyword) > > tl;dr a new `force` keyword would be better > I agree, “copy” is not a good description of this desired coercion behavior. A new keyword argument like “force” would be much clearer. > Cheers, > Ralf > > >> I think the discussion stalled on the precise spelling of the third >> option. >> >> `__array__` was not discussed there, but it seems like adding the `copy` >> argument to `__array__` would be a perfectly reasonable extension. >> >> Eric >> >> On Fri, 24 Apr 2020 at 03:00, Juan Nunez-Iglesias >> wrote: >> >>> Hi everyone, >>> >>> One bit of expressivity we would miss is “copy if necessary, but otherwise don’t bother”, but there are workarounds to this. >>> >>> After a side discussion with Stéfan van der Walt, we came up with >>> `allow_copy=True`, which would express to the downstream library that we >>> don’t mind waiting, but that zero-copy would also be ok. >>> >>> This sounds like the sort of thing that is use case driven. If enough >>> projects want to use it, then I have no objections to adding the keyword. >>> OTOH, we need to be careful about adding too many interoperability tricks >>> as they complicate the code and makes it hard for folks to determine the >>> best solution. Interoperability is a hot topic and we need to be careful >>> not put too leave behind too many experiments in the NumPy code. Do you >>> have any other ideas of how to achieve the same effect? >>> >>> >>> Personally, I don’t have any other ideas, but would be happy to hear >>> some! >>> >>> My view regarding API/experiment creep is that `__array__` is the oldest >>> and most basic of all the interop tricks and that this can be safely >>> maintained for future generations. Currently it only takes `dtype=` as a >>> keyword argument, so it is a very lean API. I think this particular use >>> case is very natural and I’ve encountered the reluctance to implicitly copy >>> twice, so I expect it is reasonably common. >>> >>> Regarding difficulty in determining the best solution, I would be happy >>> to contribute to the dispatch basics guide together with the new kwarg. I >>> agree that the protocols are getting quite numerous and I couldn’t find a >>> single place that gathers all the best practices together. But, to >>> reiterate my point: `__array__` is the simplest of these and I think this >>> keyword is pretty safe to add. >>> >>> For ease of discussion, here are the API options discussed so far, as >>> well as a few extra that I don’t like but might trigger other ideas: >>> >>> np.asarray(my_duck_array, allow_copy=True) # default is False, or None >>> -> leave it to the duck array to decide >>> np.asarray(my_duck_array, copy=True) # always copies, but, if supported >>> by the duck array, defers to it for the copy >>> np.asarray(my_duck_array, copy=‘allow’) # could take values ‘allow’, >>> ‘force’, ’no’, True(=‘force’), False(=’no’) >>> np.asarray(my_duck_array, force_copy=False, allow_copy=True) # separate >>> concepts, but unclear what force_copy=True, allow_copy=False means! >>> np.asarray(my_duck_array, force=True) >>> >>> Juan. >>> ___ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@python.org >>> https://mail.python.org/mailman/listinfo/numpy-discussion >>> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add `force=` or `copy=` kwarg to `__array__` interface
On Fri, Apr 24, 2020 at 12:35 PM Eric Wieser wrote: > Perhaps worth mentioning that we've discussed this sort of API before, in > https://github.com/numpy/numpy/pull/11897. > > Under that proposal, the api would be something like: > > * `copy=True` - always copy, like it is today > * `copy=False` - copy if needed, like it is today > * `copy=np.never_copy` - never copy, throw an exception if not possible > There's a couple of issues I see with using `copy` for __array__: - copy is already weird (False doesn't mean no), and a [bool, some_obj_or_str] keyword isn't making that better - the behavior we're talking about can do more than copying, e.g. for PyTorch it would modify the autograd graph by adding detach(), and for sparse it's not just "make a copy" (which implies doubling memory use) but it densifies which can massively blow up the memory. - I'm -1 on adding things to the main namespace (never_copy) for something that can be handled differently (like a string, or a new keyword) tl;dr a new `force` keyword would be better Cheers, Ralf > I think the discussion stalled on the precise spelling of the third option. > > `__array__` was not discussed there, but it seems like adding the `copy` > argument to `__array__` would be a perfectly reasonable extension. > > Eric > > On Fri, 24 Apr 2020 at 03:00, Juan Nunez-Iglesias > wrote: > >> Hi everyone, >> >> One bit of expressivity we would miss is “copy if necessary, but >>> otherwise don’t bother”, but there are workarounds to this. >>> >> >> After a side discussion with Stéfan van der Walt, we came up with >> `allow_copy=True`, which would express to the downstream library that we >> don’t mind waiting, but that zero-copy would also be ok. >> >> This sounds like the sort of thing that is use case driven. If enough >> projects want to use it, then I have no objections to adding the keyword. >> OTOH, we need to be careful about adding too many interoperability tricks >> as they complicate the code and makes it hard for folks to determine the >> best solution. Interoperability is a hot topic and we need to be careful >> not put too leave behind too many experiments in the NumPy code. Do you >> have any other ideas of how to achieve the same effect? >> >> >> Personally, I don’t have any other ideas, but would be happy to hear some! >> >> My view regarding API/experiment creep is that `__array__` is the oldest >> and most basic of all the interop tricks and that this can be safely >> maintained for future generations. Currently it only takes `dtype=` as a >> keyword argument, so it is a very lean API. I think this particular use >> case is very natural and I’ve encountered the reluctance to implicitly copy >> twice, so I expect it is reasonably common. >> >> Regarding difficulty in determining the best solution, I would be happy >> to contribute to the dispatch basics guide together with the new kwarg. I >> agree that the protocols are getting quite numerous and I couldn’t find a >> single place that gathers all the best practices together. But, to >> reiterate my point: `__array__` is the simplest of these and I think this >> keyword is pretty safe to add. >> >> For ease of discussion, here are the API options discussed so far, as >> well as a few extra that I don’t like but might trigger other ideas: >> >> np.asarray(my_duck_array, allow_copy=True) # default is False, or None >> -> leave it to the duck array to decide >> np.asarray(my_duck_array, copy=True) # always copies, but, if supported >> by the duck array, defers to it for the copy >> np.asarray(my_duck_array, copy=‘allow’) # could take values ‘allow’, >> ‘force’, ’no’, True(=‘force’), False(=’no’) >> np.asarray(my_duck_array, force_copy=False, allow_copy=True) # separate >> concepts, but unclear what force_copy=True, allow_copy=False means! >> np.asarray(my_duck_array, force=True) >> >> Juan. >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 163, Issue 23
On Sat, Apr 25, 2020 at 4:50 PM wrote: > Send NumPy-Discussion mailing list submissions to > numpy-discussion@python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > numpy-discussion-requ...@python.org > > You can reach the person managing the list at > numpy-discussion-ow...@python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of NumPy-Discussion digest..." > Today's Topics: > >1. Re: Feelings about type aliases in NumPy (Sebastian Berg) >2. beginner introduction to group (Tina Oberoi) >3. Re: beginner introduction to group (Robert Kern) >4. Re: Feelings about type aliases in NumPy (Stephan Hoyer) >5. Re: Feelings about type aliases in NumPy (Kevin Sheppard) > > > > -- Forwarded message -- > From: Sebastian Berg > To: numpy-discussion@python.org > Cc: > Bcc: > Date: Fri, 24 Apr 2020 13:29:59 -0500 > Subject: Re: [Numpy-discussion] Feelings about type aliases in NumPy > On Fri, 2020-04-24 at 11:10 -0700, Stefan van der Walt wrote: > > On Fri, Apr 24, 2020, at 08:45, Joshua Wilson wrote: > > > But, Stephan pointed out that it might be confusing to users for > > > objects to only exist at typing time, so we came around to the > > > question of whether people are open to the idea of including the > > > type > > > aliases in NumPy itself. Ralf's concrete proposal was to make a > > > module > > > numpy.types (or maybe numpy.typing) to hold the aliases so that > > > they > > > don't pollute the top-level namespace. The module would initially > > > contain the types > > > > That sounds very sensible. Having types available with NumPy should > > also encourage their use, especially if we can add some documentation > > around it. > > I agree, I might have a small tendency for `numpy.types` if we ever > find any usage other than direct typing that may be the better name? > > Out of curiousity, I guess `ArrayLike` would be an ABC that a > downstream project can register with? > > - Sebastian > > > > > > Stéfan > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > > -- Forwarded message -- > From: Tina Oberoi > To: numpy-discussion@python.org > Cc: > Bcc: > Date: Sat, 25 Apr 2020 10:30:38 +0530 > Subject: [Numpy-discussion] beginner introduction to group > Hi Everyone, > I am new to contributing to numpy. I have read the contributors guide and > done with the set-up. Hope to make some good contributions and also to > connect with all you great people in the numpy community. > Any suggestions and tips are always welcome. > > Thanks and Regards > Welcome, Tina! -- Inessa Pawson > > -- Forwarded message -- > From: Robert Kern > To: Discussion of Numerical Python > Cc: > Bcc: > Date: Sat, 25 Apr 2020 01:59:57 -0400 > Subject: Re: [Numpy-discussion] beginner introduction to group > On Sat, Apr 25, 2020 at 1:02 AM Tina Oberoi > wrote: > >> Hi Everyone, >> I am new to contributing to numpy. I have read the contributors guide and >> done with the set-up. Hope to make some good contributions and also to >> connect with all you great people in the numpy community. >> Any suggestions and tips are always welcome. >> > > Welcome! Do you have an idea what you would like to work on? > > -- > Robert Kern > > > > -- Forwarded message -- > From: Stephan Hoyer > To: Discussion of Numerical Python > Cc: > Bcc: > Date: Fri, 24 Apr 2020 23:40:20 -0700 > Subject: Re: [Numpy-discussion] Feelings about type aliases in NumPy > > > On Fri, Apr 24, 2020 at 11:31 AM Sebastian Berg < > sebast...@sipsolutions.net> wrote: > >> On Fri, 2020-04-24 at 11:10 -0700, Stefan van der Walt wrote: >> > On Fri, Apr 24, 2020, at 08:45, Joshua Wilson wrote: >> > > But, Stephan pointed out that it might be confusing to users for >> > > objects to only exist at typing time, so we came around to the >> > > question of whether people are open to the idea of including the >> > > type >> > > aliases in NumPy itself. Ralf's concrete proposal was to make a >> > > module >> > > numpy.types (or maybe numpy.typing) to hold the aliases so that >> > > they >> > > don't pollute the top-level namespace. The module would initially >> > > contain the types >> > >> > That sounds very sensible. Having types available with NumPy should >> > also encourage their use, especially if we can add some documentation >> > around it. >> >> I agree, I might have a small tendency for `numpy.types` if we ever >> find any usage other than direct typing that may be the better name? > > > Unless we anticipate adding a long list of type aliases (more than the > three suggested so far), I would lean towards adding ArrayLike to the top > level
Re: [Numpy-discussion] Feelings about type aliases in NumPy
Typing is for library developers more than end users. I would also worry that putting it into the top level might discourage other typing classes since it is more difficult to add to the top level than to a lower level module. np.typing seems very clear to me. On Sat, Apr 25, 2020, 07:41 Stephan Hoyer wrote: > > > On Fri, Apr 24, 2020 at 11:31 AM Sebastian Berg < > sebast...@sipsolutions.net> wrote: > >> On Fri, 2020-04-24 at 11:10 -0700, Stefan van der Walt wrote: >> > On Fri, Apr 24, 2020, at 08:45, Joshua Wilson wrote: >> > > But, Stephan pointed out that it might be confusing to users for >> > > objects to only exist at typing time, so we came around to the >> > > question of whether people are open to the idea of including the >> > > type >> > > aliases in NumPy itself. Ralf's concrete proposal was to make a >> > > module >> > > numpy.types (or maybe numpy.typing) to hold the aliases so that >> > > they >> > > don't pollute the top-level namespace. The module would initially >> > > contain the types >> > >> > That sounds very sensible. Having types available with NumPy should >> > also encourage their use, especially if we can add some documentation >> > around it. >> >> I agree, I might have a small tendency for `numpy.types` if we ever >> find any usage other than direct typing that may be the better name? > > > Unless we anticipate adding a long list of type aliases (more than the > three suggested so far), I would lean towards adding ArrayLike to the top > level NumPy namespace as np.ArrayLike. > > Type annotations are becoming an increasingly core part of modern Python > code. We should make it easy to appropriately type check functions that act > on NumPy arrays, and a top level np.ArrayLike is definitely more convenient > than np.types.ArrayLike. > > Out of curiousity, I guess `ArrayLike` would be an ABC that a >> downstream project can register with? > > > ArrayLike will be a typing Protocol, automatically recognizing attributes > like __array__ to indicate that something can be cast to an array. > > >> >> - Sebastian >> >> >> > >> > Stéfan >> > ___ >> > NumPy-Discussion mailing list >> > NumPy-Discussion@python.org >> > https://mail.python.org/mailman/listinfo/numpy-discussion >> >> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Feelings about type aliases in NumPy
On Fri, Apr 24, 2020 at 11:31 AM Sebastian Berg wrote: > On Fri, 2020-04-24 at 11:10 -0700, Stefan van der Walt wrote: > > On Fri, Apr 24, 2020, at 08:45, Joshua Wilson wrote: > > > But, Stephan pointed out that it might be confusing to users for > > > objects to only exist at typing time, so we came around to the > > > question of whether people are open to the idea of including the > > > type > > > aliases in NumPy itself. Ralf's concrete proposal was to make a > > > module > > > numpy.types (or maybe numpy.typing) to hold the aliases so that > > > they > > > don't pollute the top-level namespace. The module would initially > > > contain the types > > > > That sounds very sensible. Having types available with NumPy should > > also encourage their use, especially if we can add some documentation > > around it. > > I agree, I might have a small tendency for `numpy.types` if we ever > find any usage other than direct typing that may be the better name? Unless we anticipate adding a long list of type aliases (more than the three suggested so far), I would lean towards adding ArrayLike to the top level NumPy namespace as np.ArrayLike. Type annotations are becoming an increasingly core part of modern Python code. We should make it easy to appropriately type check functions that act on NumPy arrays, and a top level np.ArrayLike is definitely more convenient than np.types.ArrayLike. Out of curiousity, I guess `ArrayLike` would be an ABC that a > downstream project can register with? ArrayLike will be a typing Protocol, automatically recognizing attributes like __array__ to indicate that something can be cast to an array. > > - Sebastian > > > > > > Stéfan > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
