AlenkaF commented on code in PR #34883:
URL: https://github.com/apache/arrow/pull/34883#discussion_r1161913669
##########
python/pyarrow/array.pxi:
##########
@@ -3076,6 +3076,111 @@ cdef class ExtensionArray(Array):
return Array._to_pandas(self.storage, options, **kwargs)
+class FixedShapeTensorArray(ExtensionArray):
+ """
+ Concrete class for fixed shape tensor extension arrays.
+
+ Examples
+ --------
+ Define the extension type for tensor array
+
+ >>> import pyarrow as pa
+ >>> tensor_type = pa.fixed_shape_tensor(pa.int32(), [2, 2])
+
+ Create an extension array
+
+ >>> arr = [[1, 2, 3, 4], [10, 20, 30, 40], [100, 200, 300, 400]]
+ >>> storage = pa.array(arr, pa.list_(pa.int32(), 4))
+ >>> pa.ExtensionArray.from_storage(tensor_type, storage)
+ <pyarrow.lib.FixedShapeTensorArray object at ...>
+ [
+ [
+ 1,
+ 2,
+ 3,
+ 4
+ ],
+ [
+ 10,
+ 20,
+ 30,
+ 40
+ ],
+ [
+ 100,
+ 200,
+ 300,
+ 400
+ ]
+ ]
+ """
+
+ def to_numpy_ndarray(self):
+ """
+ Convert fixed shape tensor extension array to a numpy array (with
dim+1).
+ """
+ np_flat = np.asarray(self.storage.values)
+ numpy_tensor = np_flat.reshape((len(self),) + tuple(self.type.shape),
+ order='C')
Review Comment:
After couple of days to think it over, I would agree with all the comments
from Joris at the beginning of the thread.
1. The use of permutation to reorder data should (for now) be left to the
user to decide if they want to do that or not. I have added a note in the
docstrings:
https://github.com/apache/arrow/pull/34883/commits/dd8fd31f4b15ca2e0a39673ab2a338b991cd2032
2. `to_numpy_ndarray` is reshaping 1-D array so there is no need to check
the layout style as 1-D array is always both C-contiguous and F-contiguous.
3. We cannot create a F-contiguous tensor in PyArrow and there is a check
for row layout in `from_numpy_ndarray` so there is no option, currently, to
have a F-contiguous tensors in the tensor array anyways.
With this I propose to leave the code as is and maybe open an issue later to
discuss changes/features further.
I have also added a PR for the documentation of the `fixed_shape_tensor`
binding here: https://github.com/apache/arrow/pull/34957. After the binding PR
is merged I will run the code again, build the docs locally to check the html
and then mark the PR as ready for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
