Hi David,
Thank you very much for your proposal. I will read it and post comments to the discussion thread if necessary. From: David Li <[email protected]> Sent: Wednesday, February 15, 2023 5:59 AM To: dl <[email protected]> Subject: Re: [Flight] Releasing the flight stream that was created by GetFlightInfo Hi Taeyun, As promised I've posted a proposal for some features here [1][2]. Any feedback is welcomed. [1]: https://lists.apache.org/thread/247z3t06mf132nocngc1jkp3oqglz7jp [2]: https://docs.google.com/document/d/1jhPyPZSOo2iy0LqIJVUs9KWPyFULVFJXTILDfkadx2g/edit?usp=sharing Best, David On Tue, Jan 24, 2023, at 19:34, Taeyun Kim wrote: Hi, Yes, I am the author of the StackOverflow questions. :) (BTW, I’ve changed my name in my mail address from 김태연(which is made of Korean letters) to ‘Taeyun Kim’ for this mail. I will use the new one from now on.) I’m building a SQL-based Arrow Flight server, so my primary interest is Arrow Flight SQL (and the JDBC driver based on that). But I think that the functionalities that I asked for are also useful for Arrow Flight RPC. So, it would be nice that the functionalities would be implemented first in Arrow Flight RPC and Arrow Flight SQL would use the functionalities. Thanks. From: David Li <[email protected] <mailto:[email protected]> > Sent: Wednesday, January 25, 2023 9:01 AM To: dl <[email protected] <mailto:[email protected]> > Subject: Re: [Flight] Releasing the flight stream that was created by GetFlightInfo Hello, Do you happen to be the author of a related StackOverflow question [1] ? :) I replied there, but I'll reply here as well. We just didn't cover every possible use case from the start; Flight has received several proposals, and Flight SQL was extended almost immediately at the start as well. It's great to get questions like these to help see where we might need to evolve next. That said, I am working on a proposal to cover both this and a related question [2] in Flight and Flight SQL (as well as other things that I believe will be useful). So I agree these are use cases we should handle. (For instance, being able to explicitly release a stream would be useful for better mapping the JDBC/ODBC/ADBC drivers onto Flight SQL.) Just to check, are you interested in having a standardized way to do this in Flight RPC? Or in Flight SQL? [1]: https://stackoverflow.com/questions/75228227/apache-arrow-flight-releasing-the-flight-stream-that-was-created-by-getflightin [2]: https://stackoverflow.com/questions/75180268/apache-arrow-flight-getting-sorted-data-from-multiple-endpoints On Tue, Jan 24, 2023, at 18:27, 김태연 wrote: Hi, According to the Arrow Flight protocol definition, a client(consumer) can let the server generate a flight stream through a specified descriptor in GetFlightInfo. And the flight stream will be available for the duration defined by the server(a flight service). But it seems that there is no rpc message that 'releases' the flight stream that GetFlightInfo generated. Since the client has no standard way to know or control the duration of the flight stream availability, I think it is impossible to implement a reliable client application. And since the server has no standard way to know when the client is done with the flight stream, I think it is impossible to implement efficient flight stream management. Of course, the duration can be published and the method can be implemented in a non-standard way by the client and server that know each other, but a general client (like a BI application) that uses a standard wrapper - for example, Apache Arrow Flight SQL, let alone a wrapper of wrapper: Apache Arrow Flight SQL JDBC driver - is out of luck. Is there any standard way for a client(consumer) to release the flight stream that GetFlightInfo generated? If not, why did the designers choose not to support that feature? Thank you in advance.
