Hi Taeyun, As promised I've posted a proposal for some features here [1][2].
Any feedback is welcomed. [1]: https://lists.apache.org/thread/247z3t06mf132nocngc1jkp3oqglz7jp [2]: https://docs.google.com/document/d/1jhPyPZSOo2iy0LqIJVUs9KWPyFULVFJXTILDfkadx2g/edit?usp=sharing Best, David On Tue, Jan 24, 2023, at 19:34, Taeyun Kim wrote: > Hi, > > Yes, I am the author of the StackOverflow questions. :) > (BTW, I’ve changed my name in my mail address from 김태연(which is made of > Korean letters) to ‘Taeyun Kim’ for this mail. I will use the new one from > now on.) > > I’m building a SQL-based Arrow Flight server, so my primary interest is Arrow > Flight SQL (and the JDBC driver based on that). > But I think that the functionalities that I asked for are also useful for > Arrow Flight RPC. > So, it would be nice that the functionalities would be implemented first in > Arrow Flight RPC and Arrow Flight SQL would use the functionalities. > > Thanks. > > *From:* David Li <[email protected]> > *Sent:* Wednesday, January 25, 2023 9:01 AM > *To:* dl <[email protected]> > *Subject:* Re: [Flight] Releasing the flight stream that was created by > GetFlightInfo > > Hello, > > Do you happen to be the author of a related StackOverflow question [1] ? :) > > I replied there, but I'll reply here as well. We just didn't cover every > possible use case from the start; Flight has received several proposals, and > Flight SQL was extended almost immediately at the start as well. It's great > to get questions like these to help see where we might need to evolve next. > > That said, I am working on a proposal to cover both this and a related > question [2] in Flight and Flight SQL (as well as other things that I believe > will be useful). So I agree these are use cases we should handle. (For > instance, being able to explicitly release a stream would be useful for > better mapping the JDBC/ODBC/ADBC drivers onto Flight SQL.) > > Just to check, are you interested in having a standardized way to do this in > Flight RPC? Or in Flight SQL? > > [1]: > https://stackoverflow.com/questions/75228227/apache-arrow-flight-releasing-the-flight-stream-that-was-created-by-getflightin > [2]: > https://stackoverflow.com/questions/75180268/apache-arrow-flight-getting-sorted-data-from-multiple-endpoints > > On Tue, Jan 24, 2023, at 18:27, 김태연 wrote: >> Hi, >> >> >> >> According to the Arrow Flight protocol definition, a client(consumer) can >> let the server generate a flight stream through a specified descriptor in >> GetFlightInfo. And the flight stream will be available for the duration >> defined by the server(a flight service). >> >> But it seems that there is no rpc message that 'releases' the flight stream >> that GetFlightInfo generated. >> >> Since the client has no standard way to know or control the duration of the >> flight stream availability, I think it is impossible to implement a reliable >> client application. >> >> And since the server has no standard way to know when the client is done >> with the flight stream, I think it is impossible to implement efficient >> flight stream management. >> >> >> >> Of course, the duration can be published and the method can be implemented >> in a non-standard way by the client and server that know each other, but a >> general client (like a BI application) that uses a standard wrapper - for >> example, Apache Arrow Flight SQL, let alone a wrapper of wrapper: Apache >> Arrow Flight SQL JDBC driver - is out of luck. >> >> >> >> Is there any standard way for a client(consumer) to release the flight >> stream that GetFlightInfo generated? If not, why did the designers choose >> not to support that feature? >> >> >> >> Thank you in advance. >> >> >> >
