Hello there,


I have a java server (jetty, jersey) which produces a streaming arrow response 
(via a javax.ws.rs.core.StreamingOutput).


The stream of arrow is produced ‘on the fly’ and occasionally something goes 
wrong once we’ve already begun streaming. In this case we throw an exception 
which propagates up to jersey, which then closes the response normally (there 
seems to be no good way to get jersey to hand the exception back up to jetty to 
cause an abort on the response stream). We’re sending whole arrow batches, so 
the client just sees a regular arrow stream (i.e. the stream up to and 
including the last entire arrow row batch we sent) and so has no way to detect 
that an error occurred.


Maybe there is actually a way to get jetty to hang up the connection without 
proper termination (which the client can then detect) – although this does mean 
we have no ability to indicate why the error happened – but we have so far been 
unable to figure out how to do this.


Another thought was having some way in a streaming arrow response to indicate 
this kind of error inline in the stream/be able to ‘poison’ an arrow stream – 
to indicate to the client that something exceptional happened part way through 
the streaming response. Maybe some way of sending an EOS with an error flag 


There are probably other, arguably more elegant ways to handle this kind of 
thing e.g. if we can assume http2 everywhere something like 
https://http2.github.io/http2-spec/#StreamErrorHandler but it seems currently 
the tooling isn’t really there/at any rate it could be nice to have a 
transport-independent way of indicating an error.


I was wondering if anyone has dealt with anything similar to this/has any 
pointers/thoughts on whether adding to the streaming format makes sense here? 






Attachment: smime.p7s
Description: S/MIME cryptographic signature

Reply via email to