On Sun, Jun 1, 2014 at 8:39 AM, Tony Barbieri <[email protected]> wrote:
> I think it’d be about one billion times slower than querying anything >> local. :) > > > Lol, yes it is definitely slower than querying local...but when working > with distributed tasks I'm not sure how else you can track > progress/failures/results without using some sort of promise system :). > > When you asked me originally am I mainly using it for RPC, did you mean am > I using Celery for RPC or promises? I'm a bit confused I guess as to the > original question. Celery is a messaging system much like the later > example you listed above. When you send an asynchronous message, a promise > is returned. You can decide what you want to do with it. Ignore it, wait > for a result, periodically check the status, etc. I did wrap up some of > the messaging features into a simpler interface, but it's purely optional > to use it that way. The core messaging features of celery are still > available for the developers to use. > > The JSON-RPC interface is another optional interface to using our API. > This is more for interacting with additional web services and > communicating with our other offices. The "direct" api interface could > also be configured to work with our other offices if necessary. It's > basically just changing the RabbitMQ broker url :). > > I do see how writing code to appear as if it's running locally could lead > to confusion or ignorance as to what is actually happening. > > Ya, that is also why I disagreed a bit with that blog post that Marcus referenced. Someone in the comments made a similar statement, that it is the fault of the implementation doing the hiding rather that just saying RPC is bad. If it is very explicit, in the implementation, that you are doing a remote call, and not trying to mix it into behavior behind other functionality, then I don't really understand the problem with it. RPC doesn't also necessarily imply that you must wait on the return value. Again, at least not at the implementation level. Thrift, for instance, has oneway services where you fire and forget. The only thing that you wait on is the act of placing the network call. So it would only raise an exception if your connection was dead. Once the message is sent, your client no longer cares. You can have a oneway calls on each side of the RPC endpoints like sendMessage() and receiveResult() > > > On Sat, May 31, 2014 at 5:22 AM, Marcus Ottosson <[email protected]> > wrote: > >> you don’t think that retrieving or querying the status from a remote >> backend for the status of a distributed task would be performant? >> >> I think it’d be about one billion times slower than querying anything >> local. :) >> >> What I’m referring to though is the design aspect of RPC. I’ve been >> reading some rather discouraging information about it lately (this one >> <http://steve.vinoski.net/blog/2008/07/01/convenience-over-correctness/> >> sums up it up rather well, and this one >> <http://www.inspirel.com/articles/RPC_vs_Messaging.html> goes through >> their differences) and have been staying clear for the sake of finding out >> exactly what can be gained by doing something else; in this case - >> messaging. >> >> The argument basically boils down to the fact that making a local call is >> faster (by the billions) than making a remote one, and that dressing a >> remote call up to look local encourages bad design. I’ll try and >> illustrate, although I’m still looking to find exactly what those pros and >> cons are: >> >> import studiox >> def rpc_publish(asset): >> """Example of RPC hiding slow calls. Which are local and which are >> remote?""" >> path = studiox.publisher.get_path(asset) >> variant = studiox.path.dirname(path) >> >> # Perform quality checks >> assert studiox.qna(variant) >> assert not studiox.islocked(asset) >> >> >> studiox.commit(asset) >> studiox.push(asset) >> >> # Notify subscribers (database, peers) >> studiox.publish(asset) >> >> Compared to a message-based one, where each function - or “service” - is >> de-coupled, including handling of errors and distributed logging: >> >> import studiox >> def soa_publish(asset): >> path = studiox.publisher.get_path(asset) # Local >> >> studiox.messaging.Request(service='dirname', payload=path).send() # >> Remote >> variant = studiox.messaging.recv() # Blocking >> >> studiox.messaging.Push(service='asset.qna', >> payload=asset, >> reply_to='islocked').send() # Asynchronous >> def soa_islocked(asset): >> result = studiox.islocked(asset) >> if result is not None: >> studiox.messaging.Push(service='asset.commit', >> payload=asset, >> reply_to='push').send() >> else: >> studiox.messaging.Push(service='asset.error', >> payload='%s is locked' % asset).send() >> def soa_commit(asset): >> studiox.commit(asset) >> studiox.push(asset) >> >> studiox.messaging.Publish(service='log.published', >> payload=asset).send() >> >> Clearly more verbose, and this is where I suspect convenience may >> influence a design, potentially for the worse. >> >> how do you mange and monitor all of your celery workers? >> >> At first, this question stuck me as odd. But from what I gather, RabbitMQ >> acts as a broker, in which case you’re relying on existing implementation >> for features such as logging and monitoring. >> >> Ultimately, RabbitMQ (and others) are higher-level than ZeroMQ and in >> this particular example swarm.py is playing the role of RabbitMQ’s >> “server” application. >> >> So, the reason I found it odd was that, having written swarm.py, logging >> is merely an additional call from the broker to another worker; a logging >> worker. Monitoring it yet another call and so forth. At this point, both of >> those are rudimentary and aligns with existing functionality. >> >> “broker” of swarm.py >> <https://github.com/abstractfactory/labs/blob/ef788903194423c851337f49c8d202f6cb745e1f/python/messaging/chat/swarm.py#L57> >> >> Simple, unless there’s something I’m missing. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Python Programming for Autodesk Maya" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOCek-bmWcwNt%3D%2BQnT%2BYAn27G%2BVJ6Qv2UUOExQh%2BF4SaHQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOCek-bmWcwNt%3D%2BQnT%2BYAn27G%2BVJ6Qv2UUOExQh%2BF4SaHQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > -tony > > -- > You received this message because you are subscribed to the Google Groups > "Python Programming for Autodesk Maya" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/python_inside_maya/CAJhmvsSfQ%3DRLkmOT6gYnMNcwghQB8nOQZUhkKRHP6e9hS1Gqzg%40mail.gmail.com > <https://groups.google.com/d/msgid/python_inside_maya/CAJhmvsSfQ%3DRLkmOT6gYnMNcwghQB8nOQZUhkKRHP6e9hS1Gqzg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA3uin4y8iDucEnPa%3DaTyx62CMemHRp_xkHqgLX-qW_Fuw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
