I've implemented a version of client-side streaming for lists in
cthrift. It is _fully_ compatible with the Thrift TBinaryProtocol. It
should be possible to adapt these ideas to Thrift.
It works as follows:
1. Split the RPC into a send and a recieve part, so instead of
foo()
call
foo_send()
foo_recv()
2. For sending a stream, split into the part before the streamed list
(including the count), a part that can send portions of the streamed
list, and the part after the list. Thus
foo_send_stream_start(..., len_of_list)
while( !done ) {
foo_send_stream_do_stream(..., len_of_chunk, chunk);
}
foo_send_stream_end(...)
This idea can be extended to handle multiple streams
3. For receiving a stream, the _recv() is split into two functions: the
first which returns the count, and the other whcih returns sections of
the stream.
total = foo_recv_stream_len()
n = 0;
while( n != total ) {
size = foo_recv_stream_data(..., max, res);
n += size;
}
Note that max is the maximum chunk size, while size is the size actually
read. This can be less than max so that it can handle the case that only
part of the stream has been transferred.
