[
https://issues.apache.org/jira/browse/ARROW-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jorge updated ARROW-10109:
--------------------------
Description:
The goal of this issue is to support producing C Data arrays of Rust.
The use-case that motivated this issue was the possibility of running
DataFusion from Python and support moving arrays from DataFusion to
Python/Pyarray and vice-versa.
In particular, so that users can write Python UDFs that expect arrow arrays and
return arrow arrays, in the same spirit as pandas-udfs in Spark work for Pandas.
The brute-force way of writing these arrays is by converting element by element
from and to native types. The efficient way of doing it to pass the memory
address from and to each implementation, which is zero-copy.
To support the latter, we need an FFI implementation in Rust that produces and
consumes [C's Data
interface|https://arrow.apache.org/docs/format/CDataInterface.html]
was:
The goal of this issue is to support importing C Data arrays into Rust via FFI.
The use-case that motivated this issue was the possibility of running
DataFusion from Python and support moving arrays from DataFusion to
Python/Pyarray and vice-versa.
In particular, so that users can write Python UDFs that expect arrow arrays and
return arrow arrays, in the same spirit as pandas-udfs in Spark work for Pandas.
The brute-force way of writing these arrays is by converting element by element
from and to native types. The efficient way of doing it to pass the memory
address from and to each implementation, which is zero-copy.
To support the latter, we need an FFI implementation in Rust that produces and
consumes [C's Data
interface|https://arrow.apache.org/docs/format/CDataInterface.html]
> [Rust] Add support to produce a C Data interface
> ------------------------------------------------
>
> Key: ARROW-10109
> URL: https://issues.apache.org/jira/browse/ARROW-10109
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Rust
> Reporter: Jorge
> Assignee: Jorge
> Priority: Major
> Fix For: 3.0.0
>
>
> The goal of this issue is to support producing C Data arrays of Rust.
> The use-case that motivated this issue was the possibility of running
> DataFusion from Python and support moving arrays from DataFusion to
> Python/Pyarray and vice-versa.
> In particular, so that users can write Python UDFs that expect arrow arrays
> and return arrow arrays, in the same spirit as pandas-udfs in Spark work for
> Pandas.
> The brute-force way of writing these arrays is by converting element by
> element from and to native types. The efficient way of doing it to pass the
> memory address from and to each implementation, which is zero-copy.
> To support the latter, we need an FFI implementation in Rust that produces
> and consumes [C's Data
> interface|https://arrow.apache.org/docs/format/CDataInterface.html]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
