Joris Van den Bossche created ARROW-5857:
--------------------------------------------
Summary: [Python] converting multidimensional numpy arrays to
nested list type
Key: ARROW-5857
URL: https://issues.apache.org/jira/browse/ARROW-5857
Project: Apache Arrow
Issue Type: Improvement
Components: Python
Reporter: Joris Van den Bossche
Currently we only support 1-dimensional numpy arrays:
{code:python}
In [28]: pa.array([np.array([[1, 2], [3, 4]])],
type=pa.list_(pa.list_(pa.int64())))
...
ArrowInvalid: Can only convert 1-dimensional array values
{code}
So to create a nested list array, you can do that with lists of lists or object
numpy arrays with arrays as elements. We could expand this support to
multi-dimensional numpy arrays.
I am not sure we should do inference by default for this case, but at least
when specifying a nested ListType, this would be nice.
It can be an alternative way to have some support for tensors, next to an
ExtensionType (ARROW-1614 / ARROW-5819)
Related discussions:
https://lists.apache.org/thread.html/9b142c1709aa37dc35f1ce8db4e1ced94fcc4cdd96cc72b5772b373b@%3Cdev.arrow.apache.org%3E,
https://github.com/apache/arrow/issues/4802
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
