rok commented on code in PR #50203:
URL: https://github.com/apache/arrow/pull/50203#discussion_r3459300030
##########
python/pyarrow/src/arrow/python/python_to_arrow.cc:
##########
@@ -908,8 +908,20 @@ class PyListConverter : public ListConverter<T,
PyConverter, PyConverterTrait> {
Status AppendNdarray(PyObject* value) {
PyArrayObject* ndarray = reinterpret_cast<PyArrayObject*>(value);
+ OwnedRef flattened;
if (PyArray_NDIM(ndarray) != 1) {
- return Status::Invalid("Can only convert 1-dimensional array values");
+ // GH-49644: a fixed-size list (e.g. the storage of a fixed-shape tensor)
+ // can be built from a multi-dimensional array by flattening it in C
+ // order. The total number of elements must still match the list size,
+ // which the builder validates below. 0-dimensional arrays and
+ // variable-sized lists remain restricted to 1-dimensional values.
+ if (PyArray_NDIM(ndarray) < 2 || this->list_type_->id() !=
Type::FIXED_SIZE_LIST) {
+ return Status::Invalid("Can only convert 1-dimensional array values");
Review Comment:
This should probably say something like:
```suggestion
return Status::Invalid("Can only convert 1-dimensional array values
to variable sized lists array");
```
##########
python/pyarrow/src/arrow/python/python_to_arrow.cc:
##########
@@ -908,8 +908,20 @@ class PyListConverter : public ListConverter<T,
PyConverter, PyConverterTrait> {
Status AppendNdarray(PyObject* value) {
PyArrayObject* ndarray = reinterpret_cast<PyArrayObject*>(value);
+ OwnedRef flattened;
if (PyArray_NDIM(ndarray) != 1) {
- return Status::Invalid("Can only convert 1-dimensional array values");
+ // GH-49644: a fixed-size list (e.g. the storage of a fixed-shape tensor)
+ // can be built from a multi-dimensional array by flattening it in C
+ // order. The total number of elements must still match the list size,
+ // which the builder validates below. 0-dimensional arrays and
+ // variable-sized lists remain restricted to 1-dimensional values.
Review Comment:
This could be less verbose.
##########
python/pyarrow/src/arrow/python/python_to_arrow.cc:
##########
@@ -908,8 +908,20 @@ class PyListConverter : public ListConverter<T,
PyConverter, PyConverterTrait> {
Status AppendNdarray(PyObject* value) {
PyArrayObject* ndarray = reinterpret_cast<PyArrayObject*>(value);
+ OwnedRef flattened;
if (PyArray_NDIM(ndarray) != 1) {
- return Status::Invalid("Can only convert 1-dimensional array values");
+ // GH-49644: a fixed-size list (e.g. the storage of a fixed-shape tensor)
+ // can be built from a multi-dimensional array by flattening it in C
+ // order. The total number of elements must still match the list size,
+ // which the builder validates below. 0-dimensional arrays and
+ // variable-sized lists remain restricted to 1-dimensional values.
+ if (PyArray_NDIM(ndarray) < 2 || this->list_type_->id() !=
Type::FIXED_SIZE_LIST) {
+ return Status::Invalid("Can only convert 1-dimensional array values");
+ }
+ flattened.reset(PyArray_Ravel(ndarray, NPY_CORDER));
Review Comment:
Also to note here -
[PyArray_Ravel](https://numpy.org/devdocs/reference/c-api/array.html#c.PyArray_Ravel)
seems to default to C order. So we should document somewhere that no matter
the input arrays order we will return C order. (@AlenkaF please doublecheck me
here :) )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
