alamb commented on a change in pull request #12571:
URL: https://github.com/apache/arrow/pull/12571#discussion_r829386554
##########
File path: format/Flight.proto
##########
@@ -301,6 +301,36 @@ message Location {
*/
message Ticket {
bytes ticket = 1;
+ // Data representing some part of the data retrievable by the ticket.
+ //
+ // `inlined_completeness` indicates what part of the data retriavable
+ // by the ticket this represents. This is provided as an optimization for
+ // client/server applications that want to reduce latency to first result
+ // without requiring another RPC round-trip to retrieve the ticket.
applications
+ // built on top of Flight are responsible for any negotiation necessary on
whether
+ // inlining data is appropriate.
+ //
+ // The size of inlined_data is expected to be small (typically less then
1MB)
+ // and inlining too much data across tickets can run into underlying
transport
+ // limitations. Furthermore, since the data is expected to be small,
implementations
+ // are less likely to optimize for zero-copy in these cases.
+ //
+ // The inlined data is not expected to contain schema metadata. The schema
+ // should be identical to the schema provided on FlightInfo.
+ repeated FlightData inlined_data = 2;
+ enum InlinedCompleteness {
Review comment:
Since the Ticket is already an opaque data blob that is interpreted by
the application, I am not sure how much extra value adding a generic
`inlined_data_completeness` would be.
For applications that need to send `NO_INLINED_DATA` they simply omit
`inlined_data`
For applications that need to send data, they could include `inlined_data`
and could distinguish between "complete" and "sample data" somehow in the
application defined `Ticket` payload anyways.
##########
File path: format/Flight.proto
##########
@@ -301,6 +301,36 @@ message Location {
*/
message Ticket {
bytes ticket = 1;
+ // Data representing some part of the data retrievable by the ticket.
+ //
+ // `inlined_completeness` indicates what part of the data retriavable
+ // by the ticket this represents. This is provided as an optimization for
+ // client/server applications that want to reduce latency to first result
+ // without requiring another RPC round-trip to retrieve the ticket.
applications
+ // built on top of Flight are responsible for any negotiation necessary on
whether
+ // inlining data is appropriate.
+ //
+ // The size of inlined_data is expected to be small (typically less then
1MB)
+ // and inlining too much data across tickets can run into underlying
transport
+ // limitations. Furthermore, since the data is expected to be small,
implementations
+ // are less likely to optimize for zero-copy in these cases.
+ //
+ // The inlined data is not expected to contain schema metadata. The schema
+ // should be identical to the schema provided on FlightInfo.
+ repeated FlightData inlined_data = 2;
Review comment:
I am +1 to adding an optional `FlightData` field here. It would be great
to avoid having to make a subsequent `DoGet` request
##########
File path: format/Flight.proto
##########
@@ -301,6 +301,33 @@ message Location {
*/
message Ticket {
bytes ticket = 1;
+ // Data representing some part of the data retrievable by the ticket.
+ //
+ // `inlined_completeness` indicates what part of the data retriavable
+ // by the ticket this represents. This is provided as an optimization for
+ // client/server applications that want to reduce latency to first result
+ // without requiring another RPC round-trip to retrieve the ticket.
applications
+ // built on top of Flight are responsible for any negotiation necessary on
whether
+ // inlining data is appropriate.
+ //
+ // The size of inlined_data is expected to be small (typically less then
1MB)
+ // and inlining too much data across tickets can run into underlying
transport
+ // limitations. Furthermore, since the data is expected to be small,
implementations
+ // are less likely to optimize for zero-copy in these cases.
+ repeated FlightData inlined_data = 2;
+ enum InlinedCompleteness {
+ // Default is no data is inlined. An UNDEFINED value is not provided
because an enum
+ // for the data that the client isn't aware of makes the data unusable.
+ NO_INLINED_DATA = 0;
+ // The data present in inlined_data represents all data
+ // present for the ticket.
+ COMPLETE_DATA = 1;
+ // The data present for inlined_data represents only a partial
+ // sample of the data available for the ticket. No guarantees on the
+ // ordering are provided.
+ SAMPLE_DATA = 2;
Review comment:
I feel like this type of behavior could also be encoded using
application specific metadata (rather than explicitly in the Flight protocol
itself)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
