pitrou commented on issue #38676:
URL: https://github.com/apache/arrow/issues/38676#issuecomment-1808792764
> Also, we should try to find other functions within the codebase with
potentially the same issue. Basically everywhere a C++ object can wait for
threaded tasks to end (for example in Parquet or Dataset).
In addition to functions, we also want to look after destructors of
non-trivial Cython extension classes. For example `RecordBatchReader`:
```diff
diff --git a/python/pyarrow/ipc.pxi b/python/pyarrow/ipc.pxi
index fcb9eb729e..1e597f0352 100644
--- a/python/pyarrow/ipc.pxi
+++ b/python/pyarrow/ipc.pxi
@@ -659,6 +659,10 @@ cdef class RecordBatchReader(_Weakrefable):
# cdef block is in lib.pxd
+ def __dealloc__(self):
+ with nogil:
+ self.reader.reset()
+
def __iter__(self):
return self
```
This is a bit cumbersome and easy to forget. It could perhaps be good to
have a C++ template class automating this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
