Agreed. Sorry I didn't do a better job in the original comments in the proto file.
On Fri, Mar 11, 2022, 1:42 PM David Li <[email protected]> wrote: > Ah, thanks for the clarification Jacques - we should perhaps clarify that > in the spec itself. > > On Fri, Mar 11, 2022, at 16:37, Jacques Nadeau wrote: > > The intention was that a flight consumer could use any of the location > values and they produced equivalent data. Similar to the concept of block > location in the hdfs api. This could allow a single flight dataset to be > served from multiple locations (and/or protocols) and temhe consumer can > choose the optimal path of access (to maximize locality, etc). > > It was never meant to be application specified. > > On Fri, Mar 11, 2022, 1:24 PM Kyle Porter <[email protected]> wrote: > > I would second that - let's just make it part of implementing Flight SQL > in terms of how this should be interpreted. I don't have a specific opinion > on what the best interpretation would be, however. > > *Kyle Porter* > CEO > Bit Quill Technologies Inc. > Office: +1.778.331.3355 | Direct: +1.604.441.7318 | [email protected] > https://www.bitquill.com > > This email message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure, or distribution is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy > all copies of the original message. Thank you. > > > On Fri, Mar 11, 2022 at 1:21 PM David Li <[email protected]> wrote: > > > Are existing services really a concern? Implementing Flight SQL means > you're in control already, and I'd rather see a single clear specification > over a lot of toggles/flags. > > On Fri, Mar 11, 2022, at 16:08, James Duong wrote: > > I propose for Flight SQL, we make this an always existing GetSqlInfo > property, so clients can understand the server's intent. (As opposed to > mandating behavior one way or the other, which could make Flight SQL harder > to adapt to existing Flight services if we pick the wrong way). > > With regular Flight, the application will already be tuned to the server's > behavior so there's no need for metadata around this there. > > On Fri, Mar 11, 2022 at 1:02 PM Kyle Porter <[email protected]> > wrote: > > It seems like we should probably specify the behaviour for Flight SQL, at > the very least, to ensure that the behaviour is consistent, which is one of > the goals of Flight SQL. > > *Kyle Porter* > CEO > Bit Quill Technologies Inc. > Office: +1.778.331.3355 | Direct: +1.604.441.7318 | [email protected] > https://www.bitquill.com > > This email message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure, or distribution is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy > all copies of the original message. Thank you. > > > On Fri, Mar 11, 2022 at 1:01 PM David Li <[email protected]> wrote: > > > The perhaps unsatisfactory answer is that it is up to the application how > to interpret this. For instance, it could be used to indicate multiple > servers that can all handle the request, allowing the client to pick one at > its choosing (say, for load balancing, or because the client only supports > certain protocols). > > On Fri, Mar 11, 2022, at 15:58, James Duong wrote: > > This seems inconsistent. > > In this c++ performance test it looks like it just gets the first location > in every endpoint (though it gets data from every endpoint in parallel): > > https://github.com/apache/arrow/blob/dc2e0b2e44fdaa3d5ad0bb358ff8ce9db3bc7416/cpp/src/arrow/flight/flight_benchmark.cc#L297 > > The protobuf definition is a bit unclear too. It says the client can > redeem the ticket at every location, but doesn't say the client > must retrieve data at every location to get the whole dataset (so it's not > clear if this is for redundancy or partitioning): > > https://github.com/apache/arrow/blob/dc2e0b2e44fdaa3d5ad0bb358ff8ce9db3bc7416/format/Flight.proto#L283 > > The Java and C++ examples posted also do not handle the case where the > location is empty. > > On Fri, Mar 11, 2022 at 12:53 PM Rafael Telles <[email protected]> wrote: > > Hi guys, > > What is the user supposed to do with the Location list in a FlightEndpoint? > Looking at this integration test ( > https://github.com/apache/arrow/blob/dc2e0b2e44fdaa3d5ad0bb358ff8ce9db3bc7416/java/flight/flight-integration-tests/src/main/java/org/apache/arrow/flight/integration/tests/IntegrationTestClient.java#L151) > it's iterating on Location list and issuing multiple getStream() calls with > the same ticket. > > Best Regards, > > > > -- > *James Duong* > Lead Software Developer > Bit Quill Technologies Inc. > Direct: +1.604.562.6082 | [email protected] > https://www.bitquilltech.com > > > This email message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure, or distribution is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy > all copies of the original message. Thank you. > > > > > -- > *James Duong* > Lead Software Developer > Bit Quill Technologies Inc. > Direct: +1.604.562.6082 | [email protected] > https://www.bitquilltech.com > > > This email message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure, or distribution is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy > all copies of the original message. Thank you. > > > >
