sgilmore10 opened a new pull request, #41655: URL: https://github.com/apache/arrow/pull/41655
### Rationale for this change Now that the MATLAB interface has support for `arrow.tabular.RecordBatch` and `arrow.array.Array`, we should add support for the [C Data Interface](https://arrow.apache.org/docs/format/CDataInterface.html) format. The C Data Interface is based around two C struct definitions: (1) `ArrowArray` and (2) `ArrowSchema`. We should start by adding a new MATLAB class (e.g. `arrow.c.Array`) which wraps the underlying `ArrowArray` C struct. Later, we can add another new MATLAB class (e.g. `arrow.c.Schema`) which wraps the `ArrowSchema` C struct. Once we have added these two MATLAB classes, we can then add import and export functionality to share the Arrow memory between multiple language runtimes running in the same process. This would help enable workflows like sharing Arrow data between the MATLAB Interface to Arrow and `pyarrow` running within the MATLAB process via the [MATLAB interface to Python](https://www.mathworks.com/help/matlab/call-python-libraries.html)). ### What changes are included in this PR? 1. Added a new C++ proxy class called `arrow::matlab::c::proxy::Array` which wraps an `ArrowArray` `struct` pointer. This class is registered as the proxy `arrow.c.proxy.Array` in order to make it accessible to MATLAB. 2. Added a new MATLAB class called `arrow.c.Array` that has an `arrow.c.proxy.Array` instance. It has one public property named `Address`, which is a scalar `uint64`. This property is the memory address of the `ArrowArray` `struct` pointer owned by `arrow.c.proxy.Array`. ### Are these changes tested? Yes. 1. Added a new test class called `test/arrow/c/tArray.m`. 2. @kevingurney and I created a prototype for importing and exporting arrow `Array`s via the C Data Interface format [here](https://github.com/mathworks/arrow/tree/arrow-array-address). We were able to share arrow `Array`s and `RecordBatch`es between mlarrow and pyarrow. Our plan now is to submit the necessary MATLAB code incrementally. ### Are there any user-facing changes? Yes. The `arrow.c.Array` class is user-facing. However, it's only intended for "advanced" use-cases. In the future, we may add higher-level functionality on top of the C Data Interface so that users don't need to interact with it directly. **NOTE:** On destruction, `arrow.c.proxy.Array` will check to see if the `ArrowArray` has already been consumed by an importer. If not, `arrow.c.proxy.Array`'s destructor will call the `release` callback on the `ArrowArray` to avoid memory leaks. To the best of our knowledge, this is similar to the how the [Arrow PyCapsule Interface](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html) works. ### Future Directions 1. #41654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
