[
https://issues.apache.org/jira/browse/THRIFT-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648681#comment-13648681
]
Chet Murthy commented on THRIFT-1948:
-------------------------------------
I'd agree with Jens on this, but go further:
(1) I have (recent) experience implmenting a lock-service a la Chubby
using Thrift as the transport abstraction. In the Chubby design, the
server pushes updates to the client, and this is implemented by the
client maintaining a thread that is always blocked in an RPC on the
server; the server wakes up the thread when it wants to "send a
message" to the client.
This works fine, and I don't see a good reason to do something more to
support this mode of use.
(2) I feel somewhat stronger than Jens about -not- supporting some
sort of multi-RPC stream-like (or other) abstraction. This would bake
into the Thrift infrastructure the "state" associated with the
in-progress multi-RPC transfer, instead of leaving that to the user of
Thrift (the application writer). Different applications have
different needs in this space -- how can we know that what we're doing
is right for all users?
[it's similar to the case of supporting DAGs or cycles -- it adds
costs for users who only want to send trees, and that's the
overwhelmingly common case.]
This support would (probably) not support a client that wants to open
multiple connections to the same server and be able to transfer the
"large" stream over those connections in parallel. Wouldn't that be a
pity? How would aborting the transfer at the client's request happen?
(3) In any case, I agree with Jens that there doesn't seem to be any
particular reason the client needs to know the length of the stream
beforehand. A simple protocol could be designed to work similarly to
HTTP chunking.
(4) Now that I'm using Thrift in anger, I find that the fidelity of
the language-binding for the types that already exist, seems far, far,
more important than adding new types.
--chet--
> Add a stream type
> -----------------
>
> Key: THRIFT-1948
> URL: https://issues.apache.org/jira/browse/THRIFT-1948
> Project: Thrift
> Issue Type: New Feature
> Components: AS3 - Compiler, AS3 - Library, C glib - Compiler, C glib
> - Library, C# - Compiler, C# - Library, C++ - Compiler, C++ - Library, Cocoa
> - Compiler, Cocoa - Library, Compiler (General), D - Compiler, D - Library,
> Delphi - Compiler, Delphi - Library, Erlang - Compiler, Erlang - Library, Go
> - Compiler, Go - Library, Haskell - Compiler, Haskell - Library, Java -
> Compiler, Java - Library, JavaME - Compiler, JavaME - Library, JavaScript -
> Compiler, JavaScript - Library, Node.js - Compiler, Node.js - Library, OCaml
> - Compiler, OCaml - Library, Perl - Compiler, Perl - Library, PHP - Compiler,
> PHP - Library, Python - Compiler, Python - Library, Ruby - Compiler, Ruby -
> Library, Smalltalk - Compiler, Smalltalk - Library
> Reporter: Carl Yeksigian
> Assignee: Carl Yeksigian
>
> This is a proposal for an addition to the Thrift IDL, which allows for
> sending chunks of data between the server and the client without having the
> whole message in memory at the start of the communication.
> Here are two use cases where I have been thinking about the possibility of
> using streams.
> LockServer.thrift:
> {code}
> struct Update {
> 1: required string lock_handle,
> 2: required i64 owner
> }
> service LockService {
> stream<Update> updates_for(1: string prefix)
> }
> {code}
> This would allow the LockServer to push out updates that happen based on the
> prefix the client has specified, rather than the constant polling that would
> currently be required to imitate this interface.
> ManyResults.thrift:
> {code}
> service QueryProvider {
> stream<Result> run_query()
> }
> {code}
> This allows the query provider to run the query and send back the results as
> they come in, rather than having to bunch them up, or provide a way to page
> through the results to the client.
> The new keyword, "stream<T>", would indicate that there is a series of values
> typed T which would be communicated between client and server. Stream would
> have three primitives:
> {code}
> next(T)
> error(TException)
> end()
> {code}
> Protocols would be enhanced with the following methods:
> {code}
> writeStreamBegin(etype, streamid)
> writeStreamNext(streamid, streamMessageType)
> writeStreamNextEnd()
> writeStreamErrorEnd()
> etype, streamid = readStreamBegin()
> streamid, streamMessageType = readStreamNext()
> readStreamNextEnd()
> readStreamErrorEnd()
> {code}
> streamMessageType is one of the following:
> # next
> This means that the message will be of the element type.
> # error
> An exception was thrown during materialization of the stream.
> The stream is now closed.
> # end
> This means that the stream is finished.
> The stream is now closed.
> Once all streams are closed, readMessageEnd should be called. Before the
> first writeStreamNext() could be called, the message should otherwise be
> complete. Otherwise, an exception should be raised.
> It is possible that an exception will be thrown while the stream is being
> materialized; however, this can only occur inside of a service. In this case,
> error() will be called; the exception should be one of the exceptions that
> the service call would have thrown. The values that were generated before the
> exception will generally be valid, but may only have meaning if the stream is
> ended. All streams which are currently open may get the same exception.
> If the following service was defined:
> {code}
> stream<i64> random_numbers(stream<i64> max)
> {code}
> A sample session from client to server would be:
> {code}
> writeMessageBegin()
> writeStreamBegin(I64, 0)
> writeStreamNext(0, next)
> writeI64(10)
> writeStreamNextEnd()
> writeStreamNext(0, end)
> writeMessageEnd()
> {code}
> A sample session from server to client would be:
> {code}
> writeMessageBegin()
> writeStreamBegin(i64, 0)
> writeStreamNext(0, next)
> writeI64(3)
> writeStreamNextEnd()
> writeStreamNext(0, end)
> writeMessageEnd()
> {code}
> This change would not be compatible with previous versions of Thrift. Also,
> for languages which do not support this type of streaming, it could be
> translated into a list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
