haixuanTao commented on issue #27468:
URL: https://github.com/apache/arrow/issues/27468#issuecomment-1745399572
It seems that, the reason it does not work is that, the list conversion is
only applied to the first dimension of your numpy array. You can fix your
exemple by doing the following:
```python
out = pa.array(data.tolist()) # Throws error ArrowInvalid: Can only convert
1-dimensional array values
```
This is however not ideal as calling `tolist()` is quite slow if we compare
to first calling `ravel` and using numpy->pyarrow default conversion.
Check
```python
now = time.time();a = data.tolist();print((time.time() - now) * 1000)
## print: 0.0059604644775390625 ms
# And
now = time.time();a = data.ravel();print((time.time() - now) * 1000)
## print: 18.219709396362305 ms
# And further reconverting to pyarrow
now = time.time();a = pa.array(data.tolist());print((time.time() - now) *
1000)
## print: 51.234 ms
# Compared to
now = time.time();a = pa.array(data.ravel());print((time.time() - now) *
1000)
## print: 0.09 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
