What an ingenious hack! Cross-posting to thrift-dev in case some people don't read this list.
--David Joel Meyer wrote: > On Tue, Mar 24, 2009 at 5:01 PM, Doug Daniels > <[email protected]>wrote: > >> Ok I definitely plan on giving the Async RPC methods a try tonight, but I >> figured I'd just throw out some questions before I get home to start >> hacking >> on this stuff. >> >> The one-to-one message to RPC call Async solution will let a client send >> messages of any given type in my defined protocol, but how would a server >> respond to a client with a message that the client didn't request? For >> example say I was trying to write a FPS like Quake and I want to server to >> send position updates for all clients to all clients, how would i model >> that >> as a client RPC request for that. With the Async RPC solutions I could make >> a RPC call for Map<Integer, Position> getPositionUpdates(), Now say that >> the >> client needs to request 50 other messages to be notified of. I guess the >> solution would be to make an Async RPC call requesting those updates and >> respond to it when I receive it asynchronously and then reissue another >> Async RPC call for the next set of updates. It just seems inefficient to >> actively make the client request for data when the server could implicitly >> know that when connected on this game protocol I can just send these >> messages to the clients without them asking for it. Not to mention you'd >> have make sure you don't "miss" sending a client a message if they finished >> their Async call but haven't reestablished a new one. >> > > I think I've done something similar to what you're trying to do, and as long > as you can commit to using only async messages it's possible to pull it off > without having to start a server on the client to accept RPCs from the > server. > > When your RPC is marked as async the server doesn't send a response and the > client doesn't try to read one. So, if all your RPC calls from the client to > the server are async you have effectively freed up the inbound half of the > socket connection. That means that you can use it for receiving async > messages from the server - the only catch is that you have to start a new > thread to read and dispatch the incoming async RPC calls. > > In a typical Thrift RPC system you'd create a MyService.Processor on your > server and a MyService.Client on your client. To do bidirectional async > message sending you'll need to go a step further and create a > MyService.Client on your server for each client that connects (this can be > accomplished by providing your own TProcessorFactory) and then on each > client you create a MyService.Processor. (This assumes that you've gone with > a generic MyService definition like you described above that has a bunch of > optional messages, another option would be to define separate service > definitions for the client and server.) With two clients connected the > objects in existence would look something like this: > > Server: > MyService.Processor mainProcessor - handles incoming async RPCs > MyService.Client clientA - used to send outgoing async RPCs to ClientA > MyService.Client clientB - used to send outgoing async RPCs to ClientB > > ClientA: > MyService.Client - used to send messages to Server > MyService.Processor clientProcessor - used (by a separate thread) to > process incoming async RPCs > > ClientB: > MyService.Client - used to send messages to Server > MyService.Processor clientProcessor - used (by a separate thread) to > process incoming async RPCs > > Hopefully that explains the concept. If you need example code I can try and > pull something together (it will be in Java). The nice thing about this > method is that you don't have to establish two connections, so you can get > around the firewall issues others have mentioned. I've been using this > method on a service in production and haven't had any problems. When you > have a separate thread in your client running a Processor you're basically > blocking on a read, waiting for a message from the server. The benefit of > this is that you're notified immediately when the server shuts down instead > of having to wait until you send a message and then finding out that the TCP > connection was reset. > > Cheers, > Joel > > >> The biggest issue is that not all client request will result in a single >> response (like shooting a bullet, may blowup an entity, and damage all >> players in the area those events are seperate messages sent from the >> respective entities). >> >> At a game development studio I used to work at we developed a cross >> language >> IDL network protocol definition (C++, Java) very similiar to Protocol >> Buffers and Thrift (without some of the more mature features like being >> transport agnostic we explicitly built it for binary TCP socket transport, >> or protocol versioning), the stream of packets would contain as the first >> 32 >> bits a message ID that would be a key to a map a Message class that would >> have methods to read in that message type from a byte[] stream. >> >> Looking through Thrift code in the TBinaryProtocol writeMessage it looks >> like it's including the name of the message being sent and it's type (is >> the >> concept of Message in thrift the same as RPC?), if so what's the >> corresponding code pathway for the client waiting for an RPC response >> because if I could just use this message name or type to key into what I >> need to serialize off the network from both client and server end then that >> would be perfect. >> >> >> >> On Tue, Mar 24, 2009 at 1:51 PM, Ted Dunning <[email protected]> >> wrote: >> >>> I really think that using async service methods which are matched one to >>> one >>> with the message types that you want to send gives you exactly the >>> semantics >>> that are being requested with very simple implementation cost. >>> >>> It is important to not get toooo hung up on what RPC stands for. I use >>> async methods all the time to stream data structures for logging and it >>> works great. Moreover, it provides a really simple way of building >>> extractors and processors for this data since I have an interface sitting >>> there that will tell me about all of the methods (data types) that I need >>> to >>> handle or explicitly ignore. >>> >>> So the trick works and works really well. Give it a try! >>> >>> On Tue, Mar 24, 2009 at 8:23 AM, Bryan Duxbury <[email protected]> >> wrote: >>>> Optional fields are not serialized onto the wire. There is a slight >>>> performance penalty at serialization time if you have a ton of unset >>> fields, >>>> but that's it. >>>> >>>> Am I over complicating things >>>> Personally, sounds like it to me. Why do you need this streaming >> behavior >>>> or whatnot? Hotwiring the rpc stack to let you send any message you >> want >>> is >>>> going to be a ton of work and not really that much of a functionality >>>> improvement. >>>> >>>> -Bryan >>>> >>> >>> >>> -- >>> Ted Dunning, CTO >>> DeepDyve >>>
