lidavidm commented on a change in pull request #12571:
URL: https://github.com/apache/arrow/pull/12571#discussion_r829400209
##########
File path: format/Flight.proto
##########
@@ -301,6 +301,33 @@ message Location {
*/
message Ticket {
bytes ticket = 1;
+ // Data representing some part of the data retrievable by the ticket.
+ //
+ // `inlined_completeness` indicates what part of the data retriavable
+ // by the ticket this represents. This is provided as an optimization for
+ // client/server applications that want to reduce latency to first result
+ // without requiring another RPC round-trip to retrieve the ticket.
applications
+ // built on top of Flight are responsible for any negotiation necessary on
whether
+ // inlining data is appropriate.
+ //
+ // The size of inlined_data is expected to be small (typically less then
1MB)
+ // and inlining too much data across tickets can run into underlying
transport
+ // limitations. Furthermore, since the data is expected to be small,
implementations
+ // are less likely to optimize for zero-copy in these cases.
+ repeated FlightData inlined_data = 2;
+ enum InlinedCompleteness {
+ // Default is no data is inlined. An UNDEFINED value is not provided
because an enum
+ // for the data that the client isn't aware of makes the data unusable.
+ NO_INLINED_DATA = 0;
+ // The data present in inlined_data represents all data
+ // present for the ticket.
+ COMPLETE_DATA = 1;
+ // The data present for inlined_data represents only a partial
+ // sample of the data available for the ticket. No guarantees on the
+ // ordering are provided.
+ SAMPLE_DATA = 2;
Review comment:
FlightInfo lacks application specific metadata, though.
##########
File path: format/Flight.proto
##########
@@ -301,6 +301,36 @@ message Location {
*/
message Ticket {
bytes ticket = 1;
+ // Data representing some part of the data retrievable by the ticket.
+ //
+ // `inlined_completeness` indicates what part of the data retriavable
+ // by the ticket this represents. This is provided as an optimization for
+ // client/server applications that want to reduce latency to first result
+ // without requiring another RPC round-trip to retrieve the ticket.
applications
+ // built on top of Flight are responsible for any negotiation necessary on
whether
+ // inlining data is appropriate.
+ //
+ // The size of inlined_data is expected to be small (typically less then
1MB)
+ // and inlining too much data across tickets can run into underlying
transport
+ // limitations. Furthermore, since the data is expected to be small,
implementations
+ // are less likely to optimize for zero-copy in these cases.
+ //
+ // The inlined data is not expected to contain schema metadata. The schema
+ // should be identical to the schema provided on FlightInfo.
+ repeated FlightData inlined_data = 2;
+ enum InlinedCompleteness {
Review comment:
Ticket is meant to be opaque to the client, so it may not be appropriate
to use for this? (I've always thought of it as analogous to cookies or HTTP
session state - the client carries it and passes it back to the server, but
doesn't manipulate it.) That said I do agree that a general application
metadata field somewhere in FlightInfo would likely be useful.
##########
File path: format/Flight.proto
##########
@@ -301,6 +301,44 @@ message Location {
*/
message Ticket {
bytes ticket = 1;
+
+ // Data representing some part of the data retrievable by the ticket.
+ //
+ // This is an optional optimization for applications that want to reduce
latency,
+ // for previewing results or retrieving small data sets by providing data
+ // without requiring a DoGet call. Applications built on top of Flight
+ // are responsible for any negotiation necessary on whether
+ // inlining data is appropriate.
+ //
+ // The size of inlined_data is expected to be small (typically less then
1MB)
+ // and inlining too much data across tickets can run into underlying
transport
+ // limitations. Furthermore, since the data is expected to be small,
implementations
+ // are less likely to optimize for zero-copy in these cases.
+ message InlinedData {
+ // Arrow data for consumption.
+ //
+ // The data is not expected to contain schema metadata. The schema
Review comment:
I guess nit, but maybe 'not expected to contain the schema message'?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
