sjperkins commented on issue #16264:
URL: https://github.com/apache/arrow/issues/16264#issuecomment-2313088410
> I think the question here is also do we want a complex tensor extension
array or a complex extension array. I am not sure we can use an complex
extension array as storage of FixedShapeTensorArray,
I suspect this is possible because in this excerpt from the larger example
above,
```python
tt = pa.fixed_shape_tensor(ComplexFloatExtensionType(), (2,))
storage = pa.FixedSizeListArray.from_arrays(arrow_data, 2)
assert len(storage) == 2
tensor = pa.ExtensionArray.from_storage(tt, storage)
```
`arrow_data` is a Python Extension Array associated with an Python Extension
type with storage type `pa.binary(8)`. Does your question relate to the
underlying C++ Extension Type and Array?
```python
ipdb> type(arrow_data)
<class '__main__.ComplexFloatExtensionArray'>
ipdb>
```
Although I think wrapping it in a FixedSizeList is necessary for it to be
accepted by the FixedShapeTensorType which uses a FixedSizeList under the hood?
> though if we can that would be best. Can an extension array be storage to
another extension array?
The python example above might suggest it's possible, but I'm admittedly
not sure about the C++ layer.
> (Or am I misunderstanding and you mean we should introduce a primary
complex type?
I'm advocating for an C++ Extension ComplexFloatType (complex64) with an
associated Extension ComplexFloatArray. The FixedShapeTensorType should accept
the ComplexFloatType as a type and should be able to wrap the
ComplexFloatArray. This could be used for a proper conversion to NumPy and
Pandas. And the same again with ComplexDoubleType (complex128). They would be
exposed in the Python layer, much like the other canonical Extension Types.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
