On Tue, Jul 2, 2019 at 8:38 AM Stephan Hoyer <sho...@gmail.com> wrote:
> On Tue, Jul 2, 2019 at 8:16 AM Ralf Gommers <ralf.gomm...@gmail.com> > wrote: > >> >> >> On Tue, Jul 2, 2019 at 1:45 AM Juan Nunez-Iglesias <j...@fastmail.com> >> wrote: >> >>> On Tue, 2 Jul 2019, at 4:34 PM, Stephan Hoyer wrote: >>> >>> This is addressed in the NEP, see bullet 1 under "Partial implementation >>> of NumPy's API": >>> >>> http://www.numpy.org/neps/nep-0018-array-function-protocol.html#partial-implementation-of-numpy-s-api >>> >>> My concern is that fallback coercion behavior makes it difficult to >>> reliably implement "strict" overrides of NumPy's API. Fallback coercion is >>> certainly useful for interactive use, but it isn't really appropriate for >>> libraries. >>> >>> >> Do you mean "fallback coercion in NumPy itself", or "at all"? Right now >> there's lots of valid code around, e.g. `np.median(some_dask_array)`. Users >> will keep wanting to do that. Forcing everyone to write >> `np.median(np.array(some_dask_array))` serves no purpose. So the coercion >> has to be somewhere. You're arguing that that's up to Dask et al I think? >> > > Yes, I'm arguing this is up to dask to maintain backwards compatibility -- > or not, as the maintainers see fit. > > NumPy adding dispatching with __array_function__ did not break any > existing code, until the maintainers of other libraries started adding > __array_function__ methods. I hope that the risks of implementing such > experimental methods were self-evident. > Yeah, that's a bit of a chicken-and-egg story though. We add something and try to be "strict". Dask adds something because they like the idea and generally are quick to adopt these types of things. If we make it too hard to be backwards compatible, then neither NumPy nor Dask may try and it ends up breaking scikit-image & co. I for one don't care where the fix lands, but it's pretty to me that breaking scikit-image is the worst of all options. > >> Putting it in Dask right now still doesn't address Juan's backwards >> compat concern, but perhaps that could be bridged with a Dask bugfix >> release and some short-lived pain. >> > > I really think this is the best (only?) path forward. > I think I agree (depending on how easy it is to get the Dask fix landed). > > I'm not convinced that this shouldn't be fixed in NumPy though. Your >> concern "reliably implement "strict" overrides of NumPy's API" is a bit >> abstract. Overriding the _whole_ NumPy API is definitely undesirable. If >> we'd have a reference list somewhere about every function that is handled >> with __array_function__, then would that address your concern? Such a list >> could be auto-generated fairly easily. >> > > By "reliably implement strict overrides" I mean the ability to ensure that > every operation either uses an override or raises an informative error -- > making it very clear which operation needs to be implemented or avoided. > That isn't necessarily a good goal in itself though. In many cases, an `asarray` call still needs to go *somewhere*. If the "reliably implement strict overrides" is to help library authors, then there may be other ways to do that. For end users it can only hurt; those TypeErrors aren't exactly easy to understand. > It's true that we didn't really consider "always issuing warnings" as a > long term solution in the NEP. I can see how this would simply a backwards > compatibility story for libraries like dask, but in general, I really don't > like warnings: > I agree. Cheers, Ralf Using them like exceptions can easily result in code that is partially > broken or that fails later for non-obvious reasons. There's a reason why > Python's errors stop execution flow, until errors in languages like PHP or > JavaScript. > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
