raulcd commented on code in PR #46194: URL: https://github.com/apache/arrow/pull/46194#discussion_r2073213865
########## docs/source/format/Flight.rst: ########## @@ -360,6 +367,52 @@ string, so the obvious candidates are not compatible. The chosen representation can be parsed by both implementations, as well as Go's ``net/url`` and Python's ``urllib.parse``. +.. _flight-extended-uris: + +Extended Location URIs +---------------------- + +In addition to alternative transports, a server may also return +URIs that reference an external service or object storage location. +This can be useful in cases where intermediate data is cached as +Apache Parquet files on cloud storage or is otherwise accessible +via an HTTP service. In these scenarios, it is more efficient to be +able to provide a URI where the client may simply download the data +directly, rather than requiring a Flight service to read it back into +memory and serve it from a ``DoGet`` request. + +To avoid the complexities of Flight clients having to implement support +for multiple different cloud storage vendors (e.g. AWS S3, Google Cloud), +we extend the URIs to only allow an HTTP/HTTPS URI where the client can +perform a simple GET request to download the data. Authentication can be +handled either by negotiating externally to the Flight protocol or by the +server sending a presigned URL that the client can make a GET request to. +This should be supported by all current major cloud storage vendors, meaning +only the server needs to know the semantics of the underlying object store APIs. + +When using an extended location URI, the client should ignore any +value in the ``Ticket`` field of the ``FlightEndpoint``. The +``Ticket`` is only used for identifying data in the context of a +Flight service, and is not needed when the client is directly +downloading data from an external service. + +Clients should assume that, unless otherwise specified, the data is +being returned using the :ref:`ipc-streaming-format` just as it would Review Comment: Is this supposed to be a link? Is not working as seen on the latest preview docs build: http://crossbow.voltrondata.com/pr_docs/46194/format/Flight.html#flight-extended-uris `ipc-streaming-format` ref does not exist, should this be: ```suggestion being returned using the :ref:`format-ipc` just as it would ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
