[
https://issues.apache.org/jira/browse/ARROW-15282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yibo Cai updated ARROW-15282:
-----------------------------
Description:
The goal is to decouple FlightData payload transmission from gRPC and provide a
convenient framework to support other accelerated data planes (ucx, dpdk, xdp,
etc.).
As a PoC, I implemented a shared memory data plane for local rpc, the
performance is great compared with current best (grpc over unix socket).
Tested throughput and latency on xeon gold 5218, DoGet, batch size 128K.
||streams||grpc over unix socket||shared memory data plane||
|1|3324 MB/s, 35 us|7045 MB/s, 16 us|
|2|6289 MB/s, 38 us|13311 MB/s, 17 us|
|4|10037 MB/s, 44 us|25012 MB/s, 17 us|
was:
The goal is to decouple FlightData payload transmission from gRPC and provide a
convenient framework to support other accelerated data planes (ucx, dpdk, xdp,
etc.).
As a PoC, I implemented a shared memory data plane for local rpc, the
performance is great compared with current best (grpc over unix socket).
Tested on xeon gold 5218.
||streams||grpc over unix socket||shared memory data plane||
|1|3324 MB/s|7045 MB/s|
|2|6289 MB/s|13311 MB/s|
|4|10037 MB/s|25012 MB/s|
> [C++][FlightRPC] Support non-grpc data planes
> ---------------------------------------------
>
> Key: ARROW-15282
> URL: https://issues.apache.org/jira/browse/ARROW-15282
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++, FlightRPC
> Reporter: Yibo Cai
> Assignee: Yibo Cai
> Priority: Major
>
> The goal is to decouple FlightData payload transmission from gRPC and provide
> a convenient framework to support other accelerated data planes (ucx, dpdk,
> xdp, etc.).
> As a PoC, I implemented a shared memory data plane for local rpc, the
> performance is great compared with current best (grpc over unix socket).
> Tested throughput and latency on xeon gold 5218, DoGet, batch size 128K.
> ||streams||grpc over unix socket||shared memory data plane||
> |1|3324 MB/s, 35 us|7045 MB/s, 16 us|
> |2|6289 MB/s, 38 us|13311 MB/s, 17 us|
> |4|10037 MB/s, 44 us|25012 MB/s, 17 us|
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
