Thanks Justin, things are a lot clearer now.
On 18 March 2014 10:43, Justin Israel <[email protected]> wrote: > Protocol buffer is similar for part of what Thrift does but only covers > the serialization spec. It doesn't give you the RPC mechanisms. You still > have to handle that layer yourself. Its just a versioning > serialization/wire protocol. > But yes you would have to say what type a service will return to allow you > to fulfil the ability to generate code for static typed languages. Also it > is needed to guarantee the versionable wire spec since it has to be able to > encode the right sizes for types. But then you get all the free type > validation. Thrift doesn't even put it on the wire if you construct things > wrong. > > RPyc makes perfect sense if you are dealing with 100% python systems as it > probably strives to remain pythonically flexible. Thrift has to work in a > way that the IDL can generate code for different languages. I suggest you > try it out and see what you think. > On Mar 18, 2014 9:31 PM, "Marcus Ottosson" <[email protected]> > wrote: > >> Exactly, the language-agnosticism is the only reason I'm switching from >> RPyC to something else. >> >> Have you had a look at Google Protocol Buffers? It seems similar to the >> contract you speak of. You write a generic cross-language file, parse it to >> your language, in this case Python, and use it for sending and receiving >> messages. >> >> It was really great, but I couldn't get comfortable with the fact that it >> couldn't deal with arbitrary data-types. E.g. a response with a "message" >> attribute of type string, would have to return string and nothing else. >> Great in cases where the request and response are known, but for RPC, where >> some requests might return a list and others a bool.. I got into trouble. >> >> Does that sound similar to IDL and if so how does it deal with these >> cases? >> >> >> On 18 March 2014 08:04, Justin Israel <[email protected]> wrote: >> >>> ZeroMQ has concepts that Thrift doesn't have, and likewise. Thrift has >>> no concept of events via publishing-subscriber patterns, or streaming. It >>> is purely RPC, which would be equivalent to the req-rep pattern in ZeroMQ. >>> And ZeroMQ doesn't have all extra layers that actually make it an RPC >>> system. You would have to roll that functionality all yourself. But the >>> thing ZeroMQ and Thrift both give you over RPyc is language-agnostic >>> communication. >>> >>> The really attractive thing with Thrift is again that IDL language that >>> generates a contract for your RPC interfaces. It handles all of the >>> validation for you before it goes onto the wire, and does stuff like >>> Exception raising, and also a versioned approach to upgrading your API >>> without breaking older clients. If what you need is the event stuff with >>> subscribers, I don't think you will get that with Thrift. But if you really >>> do want RPC, then Thrift does most of the work for you. Just right a >>> .thrift IDL file, use the thrift tool to generate your bindings, and >>> implement the server side of the code. You can then either use the client >>> API raw, or write a thin wrapper over it. >>> >>> >>> On Tue, Mar 18, 2014 at 8:15 PM, Marcus Ottosson <[email protected] >>> > wrote: >>> >>>> My hope is to find one method of communicating across any process, both >>>> local and remote, but I'm not sure yet that it's feasible. Performance >>>> isn't an issue yet - its mainly calling os.listdir or maya.cmds.ls at >>>> the moment - but I suspect it may become in the future - producing logs, >>>> signalling subscribers of events. ZMQ seems to do it all but I've been >>>> spoiled by the simplicity of RPyC and can't bring myself to re-implement >>>> the benefits of using it in ZMQ. >>>> >>>> Thrift does sound interesting, but it also seems perpendicular to what >>>> ZMQ does, although ZMQ seems rather closer to the metal and would need more >>>> work to get going. >>>> >>>> Having worked with ZMQ before, what made you switch to Thrift? >>>> >>>> >>>> On 18 March 2014 00:15, Justin Israel <[email protected]> wrote: >>>> >>>>> I've never tried to use it for inter-thread/process communication, but >>>>> I assume it would work. I just don't know why kind of overhead it would be >>>>> in comparison with using ZeroMQ for the same solution. ZeroMQ seems a bit >>>>> lighter-weight but I guess it depends on exactly what you are doing. >>>>> >>>>> I've been using it in situations where our application server is >>>>> written in Java, and we need python, C++, and jquery clients to access it. >>>>> It makes it really nice to have a consistent API between any of the >>>>> languages that is auto-generated for you. Thrift takes care of type >>>>> checking, serializing/deserialzing, sending/receiving. >>>>> >>>>> >>>>> On Tue, Mar 18, 2014 at 10:08 AM, Marcus Ottosson < >>>>> [email protected]> wrote: >>>>> >>>>>> Ah, interesting. I've only looked briefly at Thrift. >>>>>> >>>>>> Would you recommend Thrift for inter-thread/process communication, as >>>>>> well as for communicating across languages? What other languages do you >>>>>> find it useful to bridge with RPC at the moment, and for what? >>>>>> >>>>>> >>>>>> On 17 March 2014 19:33, Justin Israel <[email protected]> wrote: >>>>>> >>>>>>> I'm not sure what inspired me specifically to do that. It just >>>>>>> seemed appropriate to make it easy to export functions as services and >>>>>>> then >>>>>>> make them discoverable when queried. I figured it was useful to be able >>>>>>> to >>>>>>> have first class registration of what is available remotely. I've >>>>>>> actually >>>>>>> never used RPyc yet. >>>>>>> >>>>>>> Currently for my RPC solutions I am working with Apache Thrift. It >>>>>>> allows you to generate bindings for all the modern languages so you can >>>>>>> have a server in one Lang and 5 clients each in different langs. It >>>>>>> handled >>>>>>> all of the network aspects and wire formats. >>>>>>> On Mar 18, 2014 8:11 AM, "Marcus Ottosson" <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> That pretty much sums up my impression of 0mq as well (having used >>>>>>>> it for about a week). >>>>>>>> >>>>>>>> About pyRpc, what inspired the use of publishing methods towards a >>>>>>>> server? Seems a common pattern in RPC implementations, perhaps that's >>>>>>>> what >>>>>>>> RPC is all about? RPyC (my only other experience) uses the same >>>>>>>> concept, >>>>>>>> except it publishes them to a separate class (a "Service") that gets >>>>>>>> passed >>>>>>>> into the server upon instantiation. What are you using for RPC >>>>>>>> currently? >>>>>>>> >>>>>>>> >>>>>>>> On 17 March 2014 18:33, Justin Israel <[email protected]>wrote: >>>>>>>> >>>>>>>>> I threw that code together when I was first learning ZeroMQ and >>>>>>>>> wanted to play with the idea of making it a simple RPC mechanism. >>>>>>>>> Turned >>>>>>>>> out some guys at RedHat were using it and asked for some more thing, >>>>>>>>> so I >>>>>>>>> expanded it a bit more. >>>>>>>>> >>>>>>>>> I really like ZeroMQ. Its a lot of lower level elements that allow >>>>>>>>> you to put together the patterns you need. While I didn't use pyRpc >>>>>>>>> very >>>>>>>>> much, I did use ZeroMQ for other things like pubsub. I think its >>>>>>>>> really >>>>>>>>> strong for a number of use cases, with RPC just being one of them. >>>>>>>>> Its good >>>>>>>>> for when you want to seamlessly scale work from interthread to >>>>>>>>> interprocess >>>>>>>>> to local network to even remote network situations, since all that >>>>>>>>> changes >>>>>>>>> is the socket types and devices you use to connect them. >>>>>>>>> >>>>>>>>> Its just a really flexible library for communication, messaging, >>>>>>>>> and concurrent processing. >>>>>>>>> On Mar 18, 2014 5:13 AM, "Marcus Ottosson" <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> This is especially targeted towards you, Justin, as I just found >>>>>>>>>> this.. >>>>>>>>>> https://github.com/justinfx/pyRpc >>>>>>>>>> >>>>>>>>>> ..as I was looking around for inspiration on how to implement the >>>>>>>>>> very same thing! >>>>>>>>>> >>>>>>>>>> How was your experience with 0mq for RPC? Are you still using >>>>>>>>>> this implementation or any like it? Are you still using 0mq? If so, >>>>>>>>>> what >>>>>>>>>> for? >>>>>>>>>> >>>>>>>>>> For everyone else, are you using 0mq? What for, and what have >>>>>>>>>> your experiences been with it so far? >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Marcus >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "Python Programming for Autodesk Maya" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/python_inside_maya/61159046-f043-4a56-b2a5-a67ef77fc178%40googlegroups.com<https://groups.google.com/d/msgid/python_inside_maya/61159046-f043-4a56-b2a5-a67ef77fc178%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Python Programming for Autodesk Maya" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA0Q3jCB6Btv45AM90BKGeMszKfzTMwyftTPRFhfPxF5Lg%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA0Q3jCB6Btv45AM90BKGeMszKfzTMwyftTPRFhfPxF5Lg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Marcus Ottosson* >>>>>>>> [email protected] >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Python Programming for Autodesk Maya" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOAxswLjnC%2BebKkx_F%2BNe4G-D6iZxdqE3E6Xb5YDkH-M6Q%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOAxswLjnC%2BebKkx_F%2BNe4G-D6iZxdqE3E6Xb5YDkH-M6Q%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Python Programming for Autodesk Maya" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA3jevShCpFOd740q0KXvAOY50x3H3cq60uYS6-94fppPw%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA3jevShCpFOd740q0KXvAOY50x3H3cq60uYS6-94fppPw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Marcus Ottosson* >>>>>> [email protected] >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Python Programming for Autodesk Maya" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOAfynu1PYkpkoecBGcOSJSNkWGFH9tdeNjqkcakHGj_mQ%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOAfynu1PYkpkoecBGcOSJSNkWGFH9tdeNjqkcakHGj_mQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Python Programming for Autodesk Maya" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA1JhAE6Kxt6LG-SvqpMCXr9ETtz%2BetaF2N2NsJ_iHzAsg%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA1JhAE6Kxt6LG-SvqpMCXr9ETtz%2BetaF2N2NsJ_iHzAsg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> *Marcus Ottosson* >>>> [email protected] >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Python Programming for Autodesk Maya" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOCgttU3zAzcRYio4kUz3EDcvNJvmL23hgqVDAheuy0VsQ%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOCgttU3zAzcRYio4kUz3EDcvNJvmL23hgqVDAheuy0VsQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Python Programming for Autodesk Maya" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA3LHL4Ak3GVmAa3aSYcsVUi57mbTRRpXp9wGiZzBAWR_g%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA3LHL4Ak3GVmAa3aSYcsVUi57mbTRRpXp9wGiZzBAWR_g%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> *Marcus Ottosson* >> [email protected] >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Python Programming for Autodesk Maya" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOA8fvgotx3L-_btMQXThpdznVNQQWuBO785-1vu3MmH7g%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOA8fvgotx3L-_btMQXThpdznVNQQWuBO785-1vu3MmH7g%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "Python Programming for Autodesk Maya" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA2yP8QvDphkp7oXzx2VVSWgEDuLJ-9ze%3D4MRsDrYxWA%2BQ%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA2yP8QvDphkp7oXzx2VVSWgEDuLJ-9ze%3D4MRsDrYxWA%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- *Marcus Ottosson* [email protected] -- You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOAsgJ-S1Znq0SiHCBS34M_JaRYa93vapBhR931pYeEMWg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
