Kksk43 opened a new issue, #40584:
URL: https://github.com/apache/arrow/issues/40584
### Describe the usage question you have. Please include as many useful
details as possible.
> The following content uses version 15.0.0 of pyarrow.
First try the following:
```
import numpy as np
import pyarrow as pa
np_dtype = np.float16
np_values = np.random.rand(2, 3, 3).astype(np_dtype)
np_values2 = np.random.rand(3, 3, 3).astype(np_dtype)
pa_values = pa.array([np_values, np_values2])
```
but error occured:
```
pa_values = pa.array([np_values, np_values2])
File "pyarrow\array.pxi", line 344, in pyarrow.lib.array
File "pyarrow\array.pxi", line 42, in pyarrow.lib._sequence_to_array
File "pyarrow\error.pxi", line 154, in
pyarrow.lib.pyarrow_internal_check_status
File "pyarrow\error.pxi", line 91, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Can only convert 1-dimensional array values
```
so I try another way:
```
pa_fix_shape_tensor = pa.FixedShapeTensorArray.from_numpy_ndarray(np_values)
pa_fix_shape_tensor2 =
pa.FixedShapeTensorArray.from_numpy_ndarray(np_values2)
pa_values = pa.array([pa_fix_shape_tensor, pa_fix_shape_tensor2])
```
another error comes up, it looks like the FixedShapeTensorArray is not
recognized:
```
pyarrow.lib.ArrowInvalid: Could not convert
<pyarrow.lib.FixedShapeTensorArray object at 0x0000023A4DA5C880>
[ ... ]
with type FixedShapeTensorArray: did not recognize Python value type when
inferring an Arrow data type
```
Finally tried to implement it indirectly using list:
```
value_list = np_values.tolist()
value_list2 = np_values2.tolist()
pa_dtype = pa.list_(pa.list_(pa.list_(pa.float16())))
pa_values = pa.array([value_list, value_list2], type=pa_dtype)
```
but it looks like there was a problem converting to float16 (I tried float32
and it worked fine):
```
...
pa_values = pa.array([value_list, value_list2], type=pa_dtype)
File "pyarrow\array.pxi", line 344, in pyarrow.lib.array
File "pyarrow\array.pxi", line 42, in pyarrow.lib._sequence_to_array
File "pyarrow\error.pxi", line 154, in
pyarrow.lib.pyarrow_internal_check_status
File "pyarrow\error.pxi", line 91, in pyarrow.lib.check_status
pyarrow.lib.ArrowTypeError: Expected np.float16 instance
```
Therefore, I want to know if there is a direct and efficient way to achieve
the requirement: Converting a list composed of multiple multi-dimensional
halffloat numpy arrays of different shapes into pyarrow.Array.
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
