extabgrad commented on PR #33925: URL: https://github.com/apache/arrow/pull/33925#issuecomment-1451636140
MATLAB’s Deep Learning Toolbox uses n-dimensional arrays which fit quite well with the proposal. It also has a special datatype called “dlarray” which is responsible for automatic differentiation. This datatype allows labelled dimensions, however, the labels are quite compatible with a permutation. We group dimension labels into categories – spatial/continuous, channel, batch, temporal, unspecified, and potential extensions – where the spatial and unspecified labels (and perhaps others in future) are allowed to be used for multiple dimensions. We find that ‘H’ and ‘W’ are very image-centric and do not extend arbitrarily, while ‘U’ might be used for additional dimensions needed for a variety of purposes in different contexts, such as filter groups, point cloud index and so forth. Consequently we need to keep track of permutations within dimensions with the same name. Therefore our main input would be to request that permutation and dim_names are not mutually exclusive. We would also like to ensure that the format will support complex data. Questions: 1. Could you clarify whether shape, dim_names and permutation are listed in left-to-right or right-to-left order. As in, is the contiguous dimension the right-most (C, python) or left-most (MATLAB, Fortran) dimension? 2. Are 1-dimensional and 0-dimensional arrays allowed? 3. “fixed_shape_array” or variations on multi-dimensional or nd array might be a more appropriate name given the mathematical implications of the term ‘tensor’. Joss Knight Development Manager GPU and Deep Learning Team MathWorks UK -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
