jorisvandenbossche commented on code in PR #33925: URL: https://github.com/apache/arrow/pull/33925#discussion_r1094418426
########## docs/source/format/CanonicalExtensions.rst: ########## @@ -72,4 +72,30 @@ same rules as laid out above, and provide backwards compatibility guarantees. Official List ============= -No canonical extension types have been standardized yet. +Fixed shape tensor +================== + +* Extension name: `arrow.fixed_shape_tensor`. + +* The storage type of the extension: ``FixedSizeList`` where: + + * **value_type** is the data type of individual tensors and + is an instance of ``pyarrow.DataType`` or ``pyarrow.Field``. + * **list_size** is the product of all the elements in tensor shape. + +* Extension type parameters: + + * **value_type** = Arrow DataType of the tensor elements + * **shape** = shape of the contained tensors as a tuple + * **is_row_major** = boolean indicating the order of elements Review Comment: > I see two problems with a "is_row_major" flag: To be clear, I am not advocating for this flag. I am advocating for removing it, and state in the spec it is always assumed to be row major. > saying that it "should be interpreted as row-major" doesn't say what the dimensions actually are Yes, and for that reason we can have the optional dimension names > The logical row-major order is given by the dimension names, so this is a bit circular? I am not fully sure what's the consequence of it being circular, but does that then mean those two indeed go together? I.e. the order of the dimension names match the data if you assume it to be row-major (if that's the case, we still need to specify this, so you know how to match the list of dimension names to the data) > So `dim_indexes` instead of `dim_names`? I don't understand how that would be useful. That would just be [0, 1, 2] for 3D tensors? It's the optional name that could give meaning to an application (is it the width of the image? or the color channel?) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
