Hi Kyle, I think at the very least, it would probably help to open a JIRA as a feature request for the use-case and maybe some sample python/rust code for how you would imagine the interaction would work.
https://github.com/apache/arrow/pull/5442 is a pull request that came out of the discussion Wes mentioned for defining a C structure that could be passed back and forth. Thanks, Micah On Thu, Sep 19, 2019 at 6:04 PM Kyle Emrick <[email protected]> wrote: > Wes, > > Thanks for getting back so quickly on this. I'll send this over to dev and > take a look at creating some issues in JIRA. I know nothing about C/C++, > but if there's something a total open source newbie can do to help, let me > know! > > On Thu, Sep 19, 2019, 11:59 AM Wes McKinney <[email protected]> wrote: > >> hi Kyle, >> >> Some development would be needed to enable the memory addresses from >> Python to be passed into a Rust-based function in process. There are >> some other issues to work out such as memory sharing semantics and how >> to allocate and hand off memory between Python/C++ and Rust. We have a >> separate thread there right now about developing a C FFI exactly for >> this purpose (so Rust could plug into the C FFI) >> >> I might suggest bringing this up on the dev@ mailing list and creating >> some JIRA issues to define the approach to making this possible, but >> in general this is the kind of thing we would like to make possible >> with the project. >> >> - Wes >> >> On Wed, Sep 18, 2019 at 9:55 PM Kyle Emrick <[email protected]> >> wrote: >> > >> > Hello! >> > >> > I am somewhat new to the Arrow world, so hopefully there's an easy >> answer I'm overlooking. I'm trying to build a simple proof of concept where >> I load some data in Python as an arrow object and then "zero copy pass" >> that to a rust function where it will be processed. I'm having a difficult >> time envisioning how this works by trying to read the docs, and I couldn't >> find any examples. It looks like I could use IPC, but I don't want to send >> anything over the network. I was attempting to do something as described in >> the following link, but arrow primitives are unsupported: >> https://developers.redhat.com/blog/2017/11/16/speed-python-using-rust/ >> > >> > Any help/insights would be much appreciated, particularly if I don't >> appear to be grasping any "big picture" concepts. >> > >> > Thank you! >> > >> > Kyle >> >
