[
https://issues.apache.org/jira/browse/ARROW-15282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yibo Cai updated ARROW-15282:
-----------------------------
Description:
The goal is to decouple FlightData payload transmission from gRPC and provide a
convenient framework to support other accelerated data planes (ucx, dpdk, xdp,
etc.).
As a PoC, I implemented a shared memory data plane for local rpc, the
performance is great compared with current best (grpc over unix socket).
Tested on xeon gold 5218.
||streams||grpc over unix socket||shared memory data plane||
|1|3324 GB/s|7045 GB/s|
|2|6289 GB/s|13311 GB/s|
|4|10037 GB/s|25012 GB/s|
was:
The goal is to decouple FlightData payload transmission from gRPC and provide a
convenient framework to support other accelerated data planes (ucx, dpdk, xdp,
etc.).
As a PoC, I implemented a shared memory data plane for local rpc, the
performance is great compared with current best (grpc over unix socket).
||streams||grpc over unix socket||shared memory data plane||
|1|3324 GB/s|7045 GB/s|
|2|6289 GB/s|13311 GB/s|
|4|10037 GB/s|25012 GB/s|
> [C++][FlightRPC] Support non-grpc data planes
> ---------------------------------------------
>
> Key: ARROW-15282
> URL: https://issues.apache.org/jira/browse/ARROW-15282
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++, FlightRPC
> Reporter: Yibo Cai
> Assignee: Yibo Cai
> Priority: Major
>
> The goal is to decouple FlightData payload transmission from gRPC and provide
> a convenient framework to support other accelerated data planes (ucx, dpdk,
> xdp, etc.).
> As a PoC, I implemented a shared memory data plane for local rpc, the
> performance is great compared with current best (grpc over unix socket).
> Tested on xeon gold 5218.
> ||streams||grpc over unix socket||shared memory data plane||
> |1|3324 GB/s|7045 GB/s|
> |2|6289 GB/s|13311 GB/s|
> |4|10037 GB/s|25012 GB/s|
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
