Yes, there's really no problem if you just want to use Flight as a pipe for Arrow data.
(There's also a proposal to inline data directly into FlightInfo, if you want to keep the separate metadata.) On Fri, Feb 3, 2023, at 17:44, Aldrin wrote: > I am planning on doing something similar, but I don't have a concrete > reference to point you to. > > In my case, I am treating the metadata as something that is re-usable (e.g. > the FlightInfo is not something that expires after the first use). > This makes the most sense when the information in FlightInfo is something > that directly names the data objects you need to retrieve. For > me, I am only expecting Flight communications to come from a library that I > have written for use by others, therefore I will have full control > of the interactions between a FlightClient and a FlightServer. > > As you point out, your main concern should probably be protocol > compatibility. If you will have control of the client side of > communications, > then I think there are minimal concerns other than how you design what a > Ticket or FlightInfo contains. > > Aldrin Montana > Computer Science PhD Student > UC Santa Cruz > > > On Fri, Feb 3, 2023 at 2:18 PM Nate Jones <nate.jo...@twosigma.com> wrote: > >> Hello, >> >> We've been using the Flight protocol similar to the way that the read path >> is described in documentation< >> https://arrow.apache.org/docs/format/Flight.html#downloading-data>. That >> is, services have a separate metadata server (at least logically separated >> such that a network round trip occurs for GetFlightInfo), which returns >> FlightInfo to be used to access data server(s). We follow a similar pattern >> for writes and exchanges, as well. >> >> While the separate metadata concept is crucial for certain applications, >> we think other use cases could be made much simpler by skipping the >> metadata step altogether - in this case, clients would craft their own >> Tickets and talk directly to data servers for reads, writes, and exchanges. >> This would be nice when we're just looking for a "normal gRPC streaming >> call but with the benefits of Flight." For example, some services have a >> metadata server that returns FlightInfo that simply points clients back to >> itself, resulting in an unnecessary round trip since the GetFlightInfo is >> essentially a “noop” here. >> >> I notice in the docs the statement "Of course, applications may ignore >> compatibility and simply treat the Flight RPC methods as low-level building >> blocks for their own purposes." Despite this, I wanted to reach out to see >> if there are any reference use cases that use Flight in this way. Are there >> any concerns that come to mind when adapting the Flight pattern like this? >> >> Thanks, >> Nate >>
