Perhaps authentication can wait until we have proper requirements? There are many authentication schemes around.
Regards Antoine. Le 14/02/2019 à 20:44, David Ming Li a écrit : > Back to extending the protocol, all we should need, and the simple thing > (IMO) to do, would be: > > - Add a `bytes data_application = 3` to FlightData > (https://github.com/apache/arrow/blob/master/format/Flight.proto#L286) > - Add a `bytes data_application = 1` to PutResult > - Change `DoPut` to `rpc DoPut(stream FlightData) returns (stream PutResult) > {}` > > The bigger question is how to change the C++/Java APIs to expose this, as > they kind of assume the only thing around is RecordBatches. > > It does sound interesting to have different underlying transports, which > would then preclude ever exposing gRPC. Would the thought then be to do > token-based authentication in DoGet/DoPut? I suppose the Ticket in DoGet and > the command in DoPut could serve that purpose. > > Best, > David > > On 2019/02/12 20:48:10, Antoine Pitrou <mailto:anto...@python.org> wrote: >> >> Hi David, >> >> I think allowing to send application-specific ancillary data in addition >> to Arrow data makes sense. >> >> (I'm also wondering whether the choice of gRPC is appropriate at all - >> the current C++ hacks around "zero-copy" are not pretty and they may not >> translate to other languages either) >> >> Regards >> >> Antoine. >> >> >> Le 12/02/2019 à 21:44, David Ming Li a écrit : >>> Hi all, >>> >>> >>> >>> We've been evaluating Flight for our use, and we're wondering if the >>> protocol is still open to extensions, as having a few application-defined >>> metadata fields would help our use cases a lot. >>> >>> >>> >>> (Apologies if this is a repost - was having issue with the spam filter.) >>> >>> >>> >>> Specifically, in DoGet, having a metadata binary blob in the server->client >>> messages would help implement resumable requests, especially as we have >>> non-monotonically-indexed data streams. This would also help us reuse >>> server-side state if we do have to resume a stream. >>> >>> >>> >>> In DoPut, we think making this call bidirectional would be useful to >>> support application-level ACKs, again to implement resumable uploads. The >>> server would thus have the opt
