On Wed, Feb 8, 2023 at 5:28 PM Sebastian Berg <sebast...@sipsolutions.net> wrote:
> On Wed, 2023-02-08 at 17:08 +0100, Francesc Alted wrote: > > On Wed, Feb 8, 2023 at 3:19 PM Sebastian Berg < > > sebast...@sipsolutions.net> > > wrote: > > > > > On Wed, 2023-02-08 at 14:31 +0100, Francesc Alted wrote: > > > > On Wed, Feb 8, 2023 at 1:42 PM Sebastian Berg < > > > > sebast...@sipsolutions.net> > > > > wrote: > > > > > > > > > On Wed, 2023-02-08 at 12:48 +0100, Francesc Alted wrote: > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > <snip> > > > > > > > > > > > Is there a way (or an ongoing effort) to express the variety > > > > > > of > > > > > > data > > > > > > types > > > > > > in NumPy that beats the above (which seems somewhat > > > > > > inconsistent > > > > > > to > > > > > > me)? > > > > > > > > > > How about using the Python buffer interface format string > > > > > (maybe > > > > > with > > > > > some limitations). > > > > > > > > > > > > > If you mean the array interface ( > > > > https://numpy.org/doc/stable/reference/arrays.interface.html), > > > > this > > > > is what > > > > dtype.str provides ( > > > > > https://numpy.org/doc/stable/reference/generated/numpy.dtype.str.html > > > > ). > > > > But the limitation here is that structured types are represented > > > > by > > > > the 'V' > > > > char, which is not properly representing it by any means. > > > > > > > > > > Ah, I was thinking of what the Python buffer protocol uses, which > > > is > > > what struct uses: > > > > > > https://docs.python.org/3/library/struct.html#module-struct > > > > > > That has some annoyances for sure, and structured dtypes with field > > > names need rather strange syntax. Also I think padding bytes at > > > best > > > are simply fields with an empty name. > > > But overall, it probably already does a better job than any `str()` > > > for > > > basic types: > > > > > > In [2]: import numpy as np > > > > > > In [3]: np.array(0, dtype="i,i,2f") > > > Out[3]: > > > array((0, 0, [0., 0.]), > > > dtype=[('f0', '<i4'), ('f1', '<i4'), ('f2', '<f4', (2,))]) > > > > > > In [4]: memoryview(np.array(0, dtype="i,i,2f")).format > > > Out[4]: 'T{i:f0:i:f1:(2)f:f2:}' > > > > > > > Aha, that's pretty cool, although I don't think this is flexible > > enough to > > support e.g. field names or nested fields. After pondering about it, > > I > > think we will add a format ID to our spec, and will stick with NumPy > > as the > > default. If in the future another format appears that is more well > > defined, we could still change the representation and use a new ID, > > while > > keeping backwards compatibility if needed. > > > > > It does support field names. I think the main problem may be that it > cannot support e.g. datetimes. You probably also can't support empty > field names (but maybe that is too weird to be useful anyway). > Right. I don't think empty field names are useful, but not supporting datetimes is a deal breaker for us. > > Not ethat in the output there `T{i:f0}` denotes that it is structured > and the name is "f0". And yes, you can nest another `T{}` inside. > Good to know. > > I am not sure whether type codes are limited to single characters > (which IMO wouldn't be nice), which (to me) would seem a bit limited. > I also think we may have to agree on e.g. an empty name denoting > padding. > While I agree that the buffer format is a good effort, I consider the NumPy representation pretty more complete (which makes sense, as it had to evolve following user's needs more closely). Still, I *hope* there will be an effort in standarizing the NumPy format a bit more formally in the future. > > - Sebastian > > > > Thanks! > > > > > > > > > > - Sebastian > > > > > > > > > > > > > > > But other than that, I don't have an obvious idea right now. > > > > > > > > > > - Sebsatian > > > > > > > > > > > > > > > > > Thanks! > > > > > > _______________________________________________ > > > > > > NumPy-Discussion mailing list -- numpy-discussion@python.org > > > > > > To unsubscribe send an email to > > > > > > numpy-discussion-le...@python.org > > > > > > > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > > > > > > Member address: sebast...@sipsolutions.net > > > > > > > > > > > > > > > _______________________________________________ > > > > > NumPy-Discussion mailing list -- numpy-discussion@python.org > > > > > To unsubscribe send an email to > > > > > numpy-discussion-le...@python.org > > > > > > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > > > > > Member address: fal...@gmail.com > > > > > > > > > > > > > > > > > _______________________________________________ > > > > NumPy-Discussion mailing list -- numpy-discussion@python.org > > > > To unsubscribe send an email to numpy-discussion-le...@python.org > > > > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > > > > Member address: sebast...@sipsolutions.net > > > > > > > > > _______________________________________________ > > > NumPy-Discussion mailing list -- numpy-discussion@python.org > > > To unsubscribe send an email to numpy-discussion-le...@python.org > > > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > > > Member address: fal...@gmail.com > > > > > > > > > _______________________________________________ > > NumPy-Discussion mailing list -- numpy-discussion@python.org > > To unsubscribe send an email to numpy-discussion-le...@python.org > > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > > Member address: sebast...@sipsolutions.net > > > _______________________________________________ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: fal...@gmail.com > -- Francesc Alted
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
