Hey Alex,
Basically, you should call start() exactly once, as you noticed, it sends the
initial schema message.
If the VectorSchemaRoot is not stable, what you should do is create your own
root with the same schema, and use VectorUnloader/VectorLoader to transfer data
from the source root to the root used by Flight.
Does that make sense? This would be good to add to the Arrow Java cookbook (at
least, the VectorLoader/Unloader part).
-David
On Mon, Feb 14, 2022, at 12:27, Alex McRae (CW) wrote:
> Hi team,
>
> We are currently building a Flight service which proxies requests in Java. We
> are currently getting getStream working on the FlightProducer.
>
> The code looks similar to this
>
> public *void* getStream(CallContext context, ServerStreamListener listener) {
> FlightStream stream = *this*.client.getStream(ticket);
> *while* (flightStream.next()) {
> *if* (!flightStream.hasRoot()) { *break*; }
>
> listener.start(flightStream.getRoot(),
> flightStream.getDictionaryProvider());
> listener.putNext();
> }
> listener.completed();
> }
>
>
> We are running into issues understanding if this is valid usage? I have
> looked at the OutBoundStreamListenerImpl.java file and it looks like calling
> start() on the listener causes it to resend some schema messages.
> We are trying to understand how to handle the case where
> flightStream.getRoot() returns a different VectorSchemaRoot than the previous
> call.
>
> For more context we have also tried
> public *void* getStream(CallContext context, ServerStreamListener listener) {
> FlightStream flightStream = *this*.client.getStream(ticket);
> listener.start(flightStream.getRoot(),
> flightStream.getDictionaryProvider());
> *while* (flightStream.next()) {
> *if* (!flightStream.hasRoot()) { *break*; }
>
> listener.putNext();
> }
> listener.completed();
> }
> But ran into issues with the connection not closing, we believe this to be
> due to the VectorSchemaRoot changing on flightStream.next() calls. We believe
> this is a issue because we are sharing the root with both the FlightStream
> and ServerStreamListener.
> https://github.com/dremio-hub/arrow-flight-client-examples is the client we
> are using to test this end to end.
>
> Please let me know if you can provide any clarity, I would be happy to update
> the documentation afterwards.
>
> Sincerely,
> Alex McRae
> [email protected]
