Unfortunately Flight wraps the C++ Flight implementation, which uses gRPC/C++, which is mostly a separate library entirely from grpcio and does not benefit from any improvements there. (The two do share a common network stack, but that's all; also, grpcio doesn't expose any of the lower level APIs that might make it possible to combine the two somehow.)
You might ask why pyarrow.flight didn't use grpcio directly (with bindings to transmute from FlightData to RecordBatch). However at the time the thought is that we would also have non-gRPC transports (which are finally being worked on) and so a from-scratch grpcio/Python implementation was not desirable. That said there are issues filed about better documenting FlightData. See ARROW-15287[1] which links a StackOverflow answer that demonstrates how to glue together asyncio/grpcio/PyArrow. There's also some previous discussion about adding async to Flight more generally [2]. [1]: https://issues.apache.org/jira/browse/ARROW-15287 [2]: "[C++] Async Arrow Flight" 2021/06/02 https://lists.apache.org/thread/jrj6yx53gyj0tr18pfdghtb8krp4gpfv -David On Tue, Feb 8, 2022, at 13:24, R KB wrote: > GRPC has pretty good AsyncIO support at this point, and since Flight is > essentially a wrapper around some GRPC types: why can't we just expose > something that generates FlightData grpc objects? > > > > >
