On Tue, Nov 29, 2011 at 00:50, Matthew Long <[email protected]>wrote:
> As a short followup -- I just looked at PyZMQ and it looks like an > excellent "pythonic" binding. In particular, it handles asynchronous calls > and queueing calls so that they can be sent via a single socket thread. > Glad you like it! The tornado IOLoop/ZMQStream stuff in zmq.eventloop should make writing a Python RPC server pretty straightforward. The model we use in IPython.parallel is essentially Async RPC-ish, and we use JSON to serialize by default, but allow drop-in replacement of other serialization methods for performance/language compatibility. I've had good experience with msgpack, and I believe protobuf works as well. -MinRK > Unfortunately, I am not doing anything in Python at the moment. > > Cheers, > > Matt > > On Nov 29, 2011, at 9:29 AM, Matthew Long wrote: > > > Hi Pieter, > > > > On Nov 28, 2011, at 10:03 PM, Pieter Hintjens wrote: > > > >> On Mon, Nov 28, 2011 at 8:36 AM, Jakub Witkowski <[email protected]> > wrote: > >> > >>> On the other hand, making a "RPC-like" system, fully customized to your > >>> application based on ZMQ is very simple; it more or less involves > >>> copy-pasting the dealer/router pattern from the examples and putting > in > >>> your business logic in. Toss in Google Protocol Buffers and suddenly > you > >>> have a full featured solution that is both easy to write and very, very > >>> fast. > >> > >> Yes, this is what I'd recommend as well. Google protobufs give you > >> language portability, are fast, and easy to use. There are various RPC > >> examples in the Guide you can start with, see Ch3 and Ch4. I'd suggest > >> either the Majordomo or Freelance patterns. > > > > Yes, those are a good start. We already use protobufs for the IDL, > service definitions and serialization. > > > > Where these examples start breaking down when I try to bring them into > my existing frameworks: > > 1) Sockets are not thread safe so we need to build scaffolding to > protect the socket. When threads are expensive like in C/C++/etc it makes > sense to do this via inproc:// connections, but in other languages (such as > Go) where go routines can be multiplexed onto any number of running threads > (by the runtime) this is an issue. > > 2) Clients often want asynchronous messages and callbacks. That is, the > client might send requests 1, 2, 3, 4, 5, but get back 5, 3, 4, 1, 2 -- and > we need to pair the request context with the correct response. > > 3) Go is a typesafe language, but we want to be able to do the type > conversion when we serialize and deserialize messages. > > > > All three are already handled in the RPC layer -- the fact that I was > reimplementing most of this is what prompted my post to the list. > > > > What I am hearing is there are no existing tools to do this and I should > implement something myself. > > > > It may be that with the appropriate surgery I can fit 0MQ in with > existing frameworks -- but this is something that the 0MQ team might want > to consider addressing at some point. While there are language bindings in > a ton of languages, a strict port of the C api probably won't be idiomatic > in the target language. If 0MQ fit in easily to the right layer of Go or > Java or Python it would be a clear win. I could use 0MQ with the native > RPC system and also implement neat patterns on top of it for PUB/SUB or > whatever. As it stands it is a little bit of a mixed win for us. > > > > If I do figure out how to cleanly add 0MQ to various RPC systems, I will > post what I did to the list. > > > > Matt > > > >> > >> -Pieter > >> _______________________________________________ > >> zeromq-dev mailing list > >> [email protected] > >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > _______________________________________________ > > zeromq-dev mailing list > > [email protected] > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
