Replacing gRPC was not the intent. The disassociated protocol is worded very generically, but works over UCX and libfabric, so it is essentially equivalent but does not force you to use the predefined Flight RPC method names so it is more flexible in that recard.
On Thu, Jul 18, 2024, at 02:20, Adam Lippai wrote: > Hi Raul, > > Finishing an experiment is good, it can help exploring more in the future > (if the community doesn’t see it as a baggage to carry forever). > > Do you have any conclusions, a summary what was learned? > > I might be wrong, but my understanding was that the initial goal was > replacing the TCP+TLS+HTTP/2+GRPC stack. The dissociated protocol handles > the IPC format. Is there anything low level focusing on the > network/transport for HPC users, data centers in the work? Or did we learn > GRPC is good enough, not the bottleneck most of the time? > > Best regards, > Adam Lippai > > > > On Wed, Jul 17, 2024 at 12:30 Raúl Cumplido <rau...@apache.org> wrote: > >> Hi, >> >> I've followed up with a PR to remove UCX transport for flight [1]. >> >> Thanks, >> Raúl >> >> [1] https://github.com/apache/arrow/pull/43297 >> >> El mié, 19 jun 2024 a las 11:29, Raúl Cumplido (<rau...@apache.org>) >> escribió: >> > >> > Hi, >> > >> > I would like to discuss deprecation of the UCX transport for Arrow >> > Flight (ARROW_WITH_UCX). >> > >> > From conversations I've had with Matt Topol and David Li: >> > - This was implemented as an experimental PoC in order to run some >> > benchmarks with flight over UCX [1] >> > - We should encourage usage of the Dissociated IPC Protocol instead of >> > that implementation [2] >> > >> > Some upstream systems are building flight with UCX and we should >> > probably not encourage its use. >> > >> > Are there any thoughts about it? >> > >> > Kind regards, >> > Raúl >> > [1] https://github.com/apache/arrow/pull/12442 >> > [2] https://arrow.apache.org/docs/dev/format/DissociatedIPC.html >>
