The intention was that a flight consumer could use any of the location values and they produced equivalent data. Similar to the concept of block location in the hdfs api. This could allow a single flight dataset to be served from multiple locations (and/or protocols) and temhe consumer can choose the optimal path of access (to maximize locality, etc).
It was never meant to be application specified. On Fri, Mar 11, 2022, 1:24 PM Kyle Porter <[email protected]> wrote: > I would second that - let's just make it part of implementing Flight SQL > in terms of how this should be interpreted. I don't have a specific opinion > on what the best interpretation would be, however. > > *Kyle Porter* > CEO > Bit Quill Technologies Inc. > Office: +1.778.331.3355 | Direct: +1.604.441.7318 | [email protected] > https://www.bitquill.com > > This email message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure, or distribution is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy > all copies of the original message. Thank you. > > > On Fri, Mar 11, 2022 at 1:21 PM David Li <[email protected]> wrote: > >> Are existing services really a concern? Implementing Flight SQL means >> you're in control already, and I'd rather see a single clear specification >> over a lot of toggles/flags. >> >> On Fri, Mar 11, 2022, at 16:08, James Duong wrote: >> >> I propose for Flight SQL, we make this an always existing GetSqlInfo >> property, so clients can understand the server's intent. (As opposed to >> mandating behavior one way or the other, which could make Flight SQL harder >> to adapt to existing Flight services if we pick the wrong way). >> >> With regular Flight, the application will already be tuned to the >> server's behavior so there's no need for metadata around this there. >> >> On Fri, Mar 11, 2022 at 1:02 PM Kyle Porter <[email protected]> >> wrote: >> >> It seems like we should probably specify the behaviour for Flight SQL, at >> the very least, to ensure that the behaviour is consistent, which is one of >> the goals of Flight SQL. >> >> *Kyle Porter* >> CEO >> Bit Quill Technologies Inc. >> Office: +1.778.331.3355 | Direct: +1.604.441.7318 | >> [email protected] >> https://www.bitquill.com >> >> This email message is for the sole use of the intended recipient(s) and >> may contain confidential and privileged information. Any unauthorized >> review, use, disclosure, or distribution is prohibited. If you are not the >> intended recipient, please contact the sender by reply email and destroy >> all copies of the original message. Thank you. >> >> >> On Fri, Mar 11, 2022 at 1:01 PM David Li <[email protected]> wrote: >> >> >> The perhaps unsatisfactory answer is that it is up to the application how >> to interpret this. For instance, it could be used to indicate multiple >> servers that can all handle the request, allowing the client to pick one at >> its choosing (say, for load balancing, or because the client only supports >> certain protocols). >> >> On Fri, Mar 11, 2022, at 15:58, James Duong wrote: >> >> This seems inconsistent. >> >> In this c++ performance test it looks like it just gets the first >> location in every endpoint (though it gets data from every endpoint in >> parallel): >> >> https://github.com/apache/arrow/blob/dc2e0b2e44fdaa3d5ad0bb358ff8ce9db3bc7416/cpp/src/arrow/flight/flight_benchmark.cc#L297 >> >> The protobuf definition is a bit unclear too. It says the client can >> redeem the ticket at every location, but doesn't say the client >> must retrieve data at every location to get the whole dataset (so it's >> not clear if this is for redundancy or partitioning): >> >> https://github.com/apache/arrow/blob/dc2e0b2e44fdaa3d5ad0bb358ff8ce9db3bc7416/format/Flight.proto#L283 >> >> The Java and C++ examples posted also do not handle the case where the >> location is empty. >> >> On Fri, Mar 11, 2022 at 12:53 PM Rafael Telles <[email protected]> wrote: >> >> Hi guys, >> >> What is the user supposed to do with the Location list in a >> FlightEndpoint? >> Looking at this integration test ( >> https://github.com/apache/arrow/blob/dc2e0b2e44fdaa3d5ad0bb358ff8ce9db3bc7416/java/flight/flight-integration-tests/src/main/java/org/apache/arrow/flight/integration/tests/IntegrationTestClient.java#L151) >> it's iterating on Location list and issuing multiple getStream() calls with >> the same ticket. >> >> Best Regards, >> >> >> >> -- >> *James Duong* >> Lead Software Developer >> Bit Quill Technologies Inc. >> Direct: +1.604.562.6082 | [email protected] >> https://www.bitquilltech.com >> >> >> This email message is for the sole use of the intended recipient(s) and >> may contain confidential and privileged information. Any unauthorized >> review, use, disclosure, or distribution is prohibited. If you are not the >> intended recipient, please contact the sender by reply email and destroy >> all copies of the original message. Thank you. >> >> >> >> >> -- >> *James Duong* >> Lead Software Developer >> Bit Quill Technologies Inc. >> Direct: +1.604.562.6082 | [email protected] >> https://www.bitquilltech.com >> >> >> This email message is for the sole use of the intended recipient(s) and >> may contain confidential and privileged information. Any unauthorized >> review, use, disclosure, or distribution is prohibited. If you are not the >> intended recipient, please contact the sender by reply email and destroy >> all copies of the original message. Thank you. >> >> >>
