Hey Taeyun, Now, here lies the issue: neither FlightEndpoint nor the FlightInfo message > in the arrow.flight.protocol.rs file of arrow-rs includes an > 'app_metadata' field (It seems outdated). Consequently, I am currently > facing a roadblock. > Is there a plan to update the flight implementation of arrow-rs?
Thanks for pointing this out, I've raised an issue on the repo for this: https://github.com/apache/arrow-rs/issues/5367 Will track progress there. Cheers, Jeffrey On Tue, 6 Feb 2024 at 13:15, 김태연 (Taeyun Kim) <taeyun....@innowireless.com> wrote: > Hi, > > I'm currently working on adding the 'DoPartialGet' functionality in my > flight client and server. > The flight client is written in Rust, while the flight server is written > in Rust and C++. > > The purpose of 'DoPartialGet' is to enable a client to request a specific > part of a flight endpoint without fully consuming it. As the official > flight protocol does not have this feature, I am implementing it using > existing methods and messages. > > The workflow of the client utilizing 'DoPartialGet' involes the following > steps: > > 1. The client initiates a query with FlightSQL on the server. > 2. The server responds with the resulting endpoints. > 3. The client treats the endpoints as an array of record arrays. > 4. The client displays the virtually concatenated records to the user page > by page, without downloading the entire content of the endpoints. > 5. To achieve this, the client invokes 'DoPartialGet' for the endpoint(s) > containing records for a particular page. > > For the 5th step, the client needs to identify which endpoint(s) hold the > records for the page. > To address this, each endpoint should report the number of records it > contains to the client in the 2nd step. > Unfortunately, the current FlightEndpoint message lacks a 'record_count' > field (It would be nice if added). > As a workaround, I decided to use the 'app_metadata' field of > FlightEndpoint message. > > Now, here lies the issue: neither FlightEndpoint nor the FlightInfo > message in the arrow.flight.protocol.rs file of arrow-rs includes an > 'app_metadata' field (It seems outdated). Consequently, I am currently > facing a roadblock. > Is there a plan to update the flight implementation of arrow-rs? > > On a related note, I'm planning to use DoExchange to implement > DoPartialGet since DoExchange allows custom information, such as offset and > count, to be sent to the server, while also allowing the server to send > FlightData. > > Thank you. > > >
