seberg opened a new issue, #40376:
URL: https://github.com/apache/arrow/issues/40376
### Describe the enhancement requested
As soon as we merge gh- in NumPy, it would be nice to have a small fix in
here. I suspect the following changes should do things (the first two are
obvious, the last is the annoying one).
The tricky remaining thing is that `itemsize_` would be nice to be an
`intp`/`(Py_)ssize_t` or `int64` now and I am not sure if that has knock-on
effects
```diff
diff --git a/python/pyarrow/src/arrow/python/arrow_to_pandas.cc
b/python/pyarrow/src/arrow/python/arrow_to_pandas.cc
index cb9cbe5b9..e29f5b31f 100644
--- a/python/pyarrow/src/arrow/python/arrow_to_pandas.cc
+++ b/python/pyarrow/src/arrow/python/arrow_to_pandas.cc
@@ -278,3 +278,3 @@ Status PyArray_NewFromPool(int nd, npy_intp* dims,
PyArray_Descr* descr, MemoryP
// * Get better performance through custom allocators
- int64_t total_size = descr->elsize;
+ int64_t total_size = PyDataType_ELSIZE(descr);
for (int i = 0; i < nd; ++i) {
diff --git a/python/pyarrow/src/arrow/python/numpy_convert.cc
b/python/pyarrow/src/arrow/python/numpy_convert.cc
index dfee88c09..f46487a57 100644
--- a/python/pyarrow/src/arrow/python/numpy_convert.cc
+++ b/python/pyarrow/src/arrow/python/numpy_convert.cc
@@ -48,3 +48,3 @@ NumPyBuffer::NumPyBuffer(PyObject* ao) : Buffer(nullptr,
0) {
data_ = const_cast<const uint8_t*>(ptr);
- size_ = PyArray_SIZE(ndarray) * PyArray_DESCR(ndarray)->elsize;
+ size_ = PyArray_NBYTES(ndarray);
capacity_ = size_;
diff --git a/python/pyarrow/src/arrow/python/numpy_interop.h
b/python/pyarrow/src/arrow/python/numpy_interop.h
index ce7baed25..1de28c20d 100644
--- a/python/pyarrow/src/arrow/python/numpy_interop.h
+++ b/python/pyarrow/src/arrow/python/numpy_interop.h
@@ -69,2 +69,7 @@
+// Backported NumPy 2 API. and can be removed if 2 is required.
+#if NPY_ABI_VERSION < 0x02000000
+ #define PyDataType_ELSIZE(descr) ((descr)->elsize)
+#endif
+
namespace arrow {
diff --git a/python/pyarrow/src/arrow/python/numpy_to_arrow.cc
b/python/pyarrow/src/arrow/python/numpy_to_arrow.cc
index 8903df31b..04572536c 100644
--- a/python/pyarrow/src/arrow/python/numpy_to_arrow.cc
+++ b/python/pyarrow/src/arrow/python/numpy_to_arrow.cc
@@ -198,3 +198,3 @@ class NumPyConverter {
length_ = static_cast<int64_t>(PyArray_SIZE(arr_));
- itemsize_ = static_cast<int>(PyArray_DESCR(arr_)->elsize);
+ itemsize_ = static_cast<int64_t>(PyArray_ITEMSIZE(arr_));
stride_ = static_cast<int64_t>(PyArray_STRIDES(arr_)[0]);
@@ -298,3 +298,3 @@ class NumPyConverter {
int64_t stride_;
- int itemsize_;
+ int64_t itemsize_;
```
---
@jorisvandenbossche do you know this would affect pandas? I am now worried
that pandas wheels will fail due to importing arrow. OTOH, it seems like you
must have solved that issue (or it is a non-issue).
### Component(s)
C++, Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
