On Thu, Nov 2, 2017 at 5:21 PM, Stephan Hoyer <sho...@gmail.com> wrote:
> On Thu, Nov 2, 2017 at 12:42 PM Nathan Goldbaum <nathan12...@gmail.com> > wrote: > >> Would this issue be ameliorated given Nathaniel's proposal to try to move >> away from subclasses and towards storing data in dtypes? Or would that just >> mean that xarray would need to ban dtypes it doesn't know about? >> > > Yes, I think custom dtypes would definitely help. Custom dtypes have a > well contained interface, so lots of operations (e.g., concatenate, > reshaping, indexing) are guaranteed to work in a dtype independent way. If > you try to do an unsupported operation for such a dtype (e.g., > np.datetime64), you will generally get a good error message about an > invalid dtype. > > In contrast, you can overload a subclass with totally arbitrary semantics > (e.g., np.matrix) and of course for duck-types as well. > > This makes a big difference for libraries like dask or xarray, which need > a standard interface to guarantee they do the right thing. I'm pretty sure > we can wrap a custom dtype ndarray with units, but there's no way we're > going to support np.matrix without significant work. It's hard to know > which is which without well defined interfaces. > Ah, but what if the dtype modifies the interface? That might sound evil, but it's something that's been proposed. For example, if I wanted to replace yt's YTArray in a backward compatibile way with a dtype and just use plain ndarrays everywhere, the dtype would need to *at least* modify ndarray's API, adding e.g. to(), convert_to_unit(), a units attribute, and several other things. Of course if I don't care about backward compatibility I can just do all of these operations on the dtype object itself. However, I suspect whatever implementation of custom dtypes gets added to numpy, it will have the property that it can act like an arbitrary ndarray subclass otherwise libraries like yt, Pint, metpy, and astropy won't be able to switch to it. -Nathan > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
