rok commented on PR #38008: URL: https://github.com/apache/arrow/pull/38008#issuecomment-3521914095
> As I understand it, the FixedShapeTensorArray has less need for a Large version, because the FixedSizeListType does not hold an offset int32 array like the ListType does. So 1 array can hold many tensors, as long as each is less than 2^31 elements. The VariableShapeTensorArray as it's currently documented can hold at most 2^31 elements in all tensors in the array combined, so that limit is hit earlier. Also, there doesn't seem to be a LargeFixedSizeListType, which has an int64 size? @pimdh Yes, that seems right. Now that some time has passed would you say there's still need for `VariableShapeTensorArray` with LargeList for `data`? We would need to amend the extension type or introduce `LargeVariableShapeTensorArray`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
