Hi Raul, Finishing an experiment is good, it can help exploring more in the future (if the community doesn’t see it as a baggage to carry forever).
Do you have any conclusions, a summary what was learned? I might be wrong, but my understanding was that the initial goal was replacing the TCP+TLS+HTTP/2+GRPC stack. The dissociated protocol handles the IPC format. Is there anything low level focusing on the network/transport for HPC users, data centers in the work? Or did we learn GRPC is good enough, not the bottleneck most of the time? Best regards, Adam Lippai On Wed, Jul 17, 2024 at 12:30 Raúl Cumplido <rau...@apache.org> wrote: > Hi, > > I've followed up with a PR to remove UCX transport for flight [1]. > > Thanks, > Raúl > > [1] https://github.com/apache/arrow/pull/43297 > > El mié, 19 jun 2024 a las 11:29, Raúl Cumplido (<rau...@apache.org>) > escribió: > > > > Hi, > > > > I would like to discuss deprecation of the UCX transport for Arrow > > Flight (ARROW_WITH_UCX). > > > > From conversations I've had with Matt Topol and David Li: > > - This was implemented as an experimental PoC in order to run some > > benchmarks with flight over UCX [1] > > - We should encourage usage of the Dissociated IPC Protocol instead of > > that implementation [2] > > > > Some upstream systems are building flight with UCX and we should > > probably not encourage its use. > > > > Are there any thoughts about it? > > > > Kind regards, > > Raúl > > [1] https://github.com/apache/arrow/pull/12442 > > [2] https://arrow.apache.org/docs/dev/format/DissociatedIPC.html >
