I re-factored my TEndpoint implementation again over the weekend and this morning, and now I'm happy enough with it to put it in a box. Once I get approval to push out the single file code generator into open source then I will move aggressively on this.
I have maintained code compatibility with previously written clients, i.e. in the C# runtime the TServer classes still exist and can be used for one-way, single-request communication. Existing code should compile and be backwards compatible with older generated clients or servers. If you take an existing Thrift IDL from a service and recompile it, if will have a default channel ID of 0, and you can then add it to the updated service with the new runtime and it will process older client's requests. Same thing for a newer client talking to an older server on channel 0. The end result of this work is threefold: 1. Channels allow multiple thrift services to be hosted on a single port. 2. TEndpoint replaces TServer and allows for two-way communications. 3. TEndpoint also allows multiple concurrent request processing on both ends. If the channel id is in the IDL or not, I guess it really doesn't matter either way. I envision it being used that way, but I can understand if the general consensus is to make the channel id configurable at runtime. I'm a newcomer here, and know much less about how Thrift is used by others. Thanks, Jim -----Original Message----- From: Bryan Duxbury [mailto:br...@rapleaf.com] Sent: Friday, May 14, 2010 5:37 PM To: thrift-dev@incubator.apache.org Subject: Re: Proposal: Channels This is a pretty interesting idea. I like how you've made two-way communication a first class construct. It does seem like this conflates two-way communication with multiplexing services. I don't think that the channel num should make it into the IDL. That would limit your ability to provide services defined by a diverse set of users from one server, since you'd need to coordinate who gets what channel number. If you're going to do that, you might as well just use port numbers. Instead, if you made the channel configurable, you could let that be defined by the server and published to the users. It'd also let you do things like provide two different channels that serve the same interface but perhaps backed by different configurations. -Bryan On Fri, May 14, 2010 at 2:23 PM, James King <james_...@dell.com> wrote: > Channels will enable Thrift to support the following features: > > * Facilitate two-way communication - either end can serve or call on a > channel. > Currently one end must be the client, the other must be the server > > * Service multiple thrift services on a single connection. > Currently only one service is allowed on a connection. > > The final result will end up with new TEndpoint concept that will pick > message headers off the wire and inspect them. Given their channel ID and > their message type, they will be routed to the appropriate request handler > or client waiting for a response. > > Initially I am doing this in the C# runtime. Channels are not part of the > current message protocol specification, so let us look at a little message > header history. There was an original binary protocol that looked like > this: > > begin > length 4 : length of function name [signed] > length n : function name > length 1 : message type > length 4 : sequence ID > end > > When versioning was introduced, the upper bit of the first I32 was turned > on so that all clients could do a signed comparison on the incoming size. > If less than zero, it is a versioned header, otherwise it is an old > non-versioned header. The new versioned header we all know today is: > > begin > length 4 : version which is: > 0x80000000 (high bit set) + > 0x7FFF0000 mask of the actual version + > 0x000000FF mask of the message type > length n : string [function name] > length 4 : sequence ID > end > > There is an empty byte in the version field which I would like to assign to > a channel ID. All existing clients today send 0x00 in this byte today. > Here is how the new VERSION_1 header would look like - note it would be > fully backwards compatible (on channel 0): > > begin > length 2 : 0x8000 + version = VERSION_1 (0x8001) > length 1 : channel id > length 1 : message type > length n : string [function name] > length 4 : sequence ID > end > > To maintain backwards compatibility, Channel 0 will be reserved for legacy > processing. In the C# runtime, the existing TServer classes will run > unchanged because they implement today's server-only, "channel 0" behavior. > > Because the Channel ID is now an integral part of the service being hosted, > I also recommend modifying the Thrift IDL for services. Before it looked > like this: > > > [14] Service ::= 'service' Identifier ( 'extends' Identifier )? '{' Function* '}' > > I would recommend: > > [14] Service ::= 'service' Identifier > ( 'channel' IntConstant )? > ( 'extends' Identifier )? > '{' Function* '}' > > If the channel is not specified, it is set to zero, which maintains > compatibility with existing schemas and runtimes. If an older parser tries > to parse a newer service with a channel optional argument it will fail, so > that is safe too. > > The existing C# runtime code still works on Channel 0, however there is a > new set of objects to replace the TServer and its variants. These are now > variants of TEndpoint. Essentialy what happens is: > > * A read pump reads off the TTransport and routes requests and replies. > > * Client makes a TConnectedEndpoint on a TSocket (or other TTransport) > Client fires up a read pump to read off the socket > > * Server makes a TListeningEndpoint on a TServerSocket (or other > TServerTransport) > Server fires up a read pump to listen to the socket > When server receives a connection, it creates a TConnectedEndpoint for it > That TConnectedEndpoint is the same type of thing that the client fired > up. > > As you can see now both sides of the connection have functional parity; a > TConnectedEndpoint is talking to another TConnectedEndpoint and both are > able to service calls as well as replies. In addition to this I made this > thread-safe so that a single TConnectedEndpoint can be used by multiple > threads simultaneously. Although each call to a client function is a > synchronous send/recv; you can have multiple threads all using the calls > because the channel ID and sequence ID are unique and can route the reply to > the correct waiting code. > > Here is a sample of the C# test code that performs two-way channel > communication: > > A service called echo.thrift is defined: > namespace csharp Test > service EchoServer channel 3 > { > string Echo (1:string toEcho), > string ReverseEcho(1:string toReverse), > } > > A service called rand.thrift is also defined: > namespace csharp Test > service RandServer channel 55 > { > i32 Random(), > } > > The following code creates a server: > > TServerSocket tServerSocket = new TServerSocket(port); > m_server = new TListeningEndpoint(tServerSocket, 255); // semaphore 255 > simultaneous > m_server.AddRequestProcessor(new EchoServer.Processor(new > EchoServerImpl())); > m_server.AddRequestProcessor(new RandServer.Processor(new > RandServerImpl())); > m_server.ClientConnect += new EventHandler(m_server_ClientConnect); > m_server.Start(); > > The following code creates the client: > > TSocket originator = new TSocket("localhost", port); > m_clientEnd = new TConnectedEndpoint(originator); > m_clientEnd.AddRequestProcessor(new EchoServer.Processor(new > EchoServerImpl())); > m_clientEnd.AddRequestProcessor(new RandServer.Processor(new > RandServerImpl())); > m_clientEnd.Start(); > > When the client connects to the server, the ClientConnect event fires so > that the test can cache the TConnectedEndpoint representing the server's end > of the connection. > > EchoServer.Client echoRequestFromClient = new > EchoServer.Client(m_clientEnd); > string echoedByServer = echoRequestFromClient.Echo("echoedByServer"); > > That calls the Echo function from the client to the server. > > EchoServer.Client echoRequestFromServer = new > EchoServer.Client(m_serverEnd); > string echoedByClient = echoRequestFromServer.Echo("echoedByClient"); > > That calls the Echo function from the server to the client - on the same > connection! > > Comments? > > James E. King, III 300 Innovative Way, Suite 301 > Senior Software Engineer Nashua, NH 03062 > Dell (EqualLogic) HIT Team (ASM) (603) 589-5895 > >
