pitrou edited a comment on pull request #8287: URL: https://github.com/apache/arrow/pull/8287#issuecomment-700170099
Having `FFI_ArrowSchema` and `FFI_ArrowArray` is fine, however they should probably minimally mirror the C struct and nothing else. In addition to those two low-level structs, you need specific structs for importing and exporting. **When exporting**, you need an allocated struct that gets stored in the `void* private_data`, and that will store ownership data (for example, a reference count). Your `release` callback will then dereferences the `private_data` pointer and release ownership (for example, by decrementing the reference count). You can see the C++ array release callback here: https://github.com/apache/arrow/blob/master/cpp/src/arrow/c/bridge.cc#L482-L502 Note the last line: ```c++ delete reinterpret_cast<ExportedArrayPrivateData*>(array->private_data); ``` This is taking the `private_data`, interpreting it as a pointer to the `ExportedArrayPrivateData` struct, and destroying it. The `ExportedArrayPrivateData` contains a [reference-counted pointer](https://github.com/apache/arrow/blob/master/cpp/src/arrow/c/bridge.cc#L475) to `ArrayData`, which is the actual C++ Arrow class containing the data. **When importing**, you need a struct that contains the `FFI_ArrowArray` (if importing an array). That struct needs to be kept alive by the corresponding Rust Arrow Array (for example, in C++ we have a `Buffer` class that can be subclassed for different kinds of buffers: allocated by C++, allocated by Python...) (*). When the struct's Rust destructor is called, it should call the embedded `FFI_ArrowArray`'s `release` callback. If you choose to manage ownership through buffers, since an array will have several buffers you probably want your importing struct to reference-count the `FFI_ArrowArray` (so that it will be released when all buffers are destroyed). I've found this example that might be useful: https://users.rust-lang.org/t/how-to-safely-deal-with-c-uint8-t-or-char-in-rust/43109/6 You can see the exact C++ equivalent here: https://github.com/apache/arrow/blob/master/cpp/src/arrow/c/bridge.cc#L1144-L1177 * the `ImportedArrayData` is a simple wrapper struct around `ArrowArray` with an additional destructor that calls the release callback * the `ImportedBuffer` has a reference-counted pointer to `ImportedArrayData`, such that the last buffer depending on a `ImportedArrayData` will release it when it disappears Please tell me if that's clear, or don't hesitate to ask more questions :-) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
