aboderinsamuel commented on code in PR #50203:
URL: https://github.com/apache/arrow/pull/50203#discussion_r3467929517
##########
python/pyarrow/src/arrow/python/python_to_arrow.cc:
##########
@@ -908,13 +908,32 @@ class PyListConverter : public ListConverter<T,
PyConverter, PyConverterTrait> {
Status AppendNdarray(PyObject* value) {
PyArrayObject* ndarray = reinterpret_cast<PyArrayObject*>(value);
- if (PyArray_NDIM(ndarray) != 1) {
- return Status::Invalid("Can only convert 1-dimensional array values");
- }
if (PyArray_ISBYTESWAPPED(ndarray)) {
// TODO
return Status::NotImplemented("Byte-swapped arrays not supported");
}
+ OwnedRef flattened;
+ if (PyArray_NDIM(ndarray) != 1) {
+ // GH-49644: a fixed-size list (e.g. fixed-shape-tensor storage) can be
+ // built from a multi-dimensional array, always flattened in C order
+ // regardless of the input's memory layout.
+ if (PyArray_NDIM(ndarray) < 2 || this->list_type_->id() !=
Type::FIXED_SIZE_LIST) {
+ return Status::Invalid(
+ "Can only convert 1-dimensional array values to a variable-sized
list");
+ }
+ // Get an aligned, C-contiguous array (copying only if needed), then view
+ // it as 1-D so its values can be read directly in C order.
+ PyObject* contiguous =
+ PyArray_CheckFromAny(value, nullptr, /*min_depth=*/0,
/*max_depth=*/0,
+ NPY_ARRAY_C_CONTIGUOUS | NPY_ARRAY_ALIGNED,
nullptr);
+ RETURN_IF_PYERROR();
+ flattened.reset(
+ PyArray_Ravel(reinterpret_cast<PyArrayObject*>(contiguous),
NPY_CORDER));
Review Comment:
The typed fast path could read PyArray_DATA directly, but the dtype-mismatch
fallback just below (value_converter_->Extend) walks the array
element-by-element and needs it 1-D. Since the array is already C-contiguous,
the ravel is a zero-copy reshape, so it's basically free and keeps a single
shared path. Happy to switch if you'd prefer, but I'd lean to keeping it. What
do you think 🙂.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
