jorisvandenbossche commented on PR #702: URL: https://github.com/apache/arrow-adbc/pull/702#issuecomment-1568232032
Related to the lifetime management inline discussion (https://github.com/apache/arrow-adbc/pull/702#discussion_r1204556304): > Keeping the Python object alive from the capsule would probably be harder? I think this should be possible relatively easily using the capsule's context something like: ```diff # adding this to the destructor @@ -469,9 +469,13 @@ cdef void pycapsule_stream_deleter(object stream_capsule): if stream.release != NULL: stream.release(stream) + parent = cpython.PyCapsule_GetContext(stream_capsule) + cpython.Py_DECREF(<object>parent) # adding this where creating the capsule @@ -512,9 +516,12 @@ cdef class ArrowArrayStreamHandle: return <uintptr_t> &self.stream def _to_capsule(self): - return cpython.PyCapsule_New( + capsule = cpython.PyCapsule_New( &self.stream, 'arrowarraystream', pycapsule_stream_deleter ) + cpython.PyCapsule_SetContext(capsule, <void *>self) + cpython.Py_INCREF(self) + return capsule ``` However, I am not sure how much this solves (apart from not needing to malloc the stream struct). Assuming that a typical consumer of a capsule object will _move_ the stream, they would still need to keep the capsule python object alive for the purpose of ensuring the statement/connection outlives the stream (assuming that the python object referenced by the capsule would keep those alive). So either way, the consumer needs to keep a reference to some python object (either the capsule, or the connection/statement of which a method returned a capsule) > For here: we could wrap the stream in a new stream that delegates to the original, and keeps a strong reference to the Python object? @lidavidm what do you mean with "wrapping" here exactly? Creating a new, actual `ArrowArrayStream*` struct (and it would keep a strong reference to the python object in the `private_data`?), or creating a python wrapper object (although that would be similar to the existing `ArrowArrayStreamHandle`, so probably not this)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
