> > I think it’d be about one billion times slower than querying anything > local. :)
Lol, yes it is definitely slower than querying local...but when working with distributed tasks I'm not sure how else you can track progress/failures/results without using some sort of promise system :). When you asked me originally am I mainly using it for RPC, did you mean am I using Celery for RPC or promises? I'm a bit confused I guess as to the original question. Celery is a messaging system much like the later example you listed above. When you send an asynchronous message, a promise is returned. You can decide what you want to do with it. Ignore it, wait for a result, periodically check the status, etc. I did wrap up some of the messaging features into a simpler interface, but it's purely optional to use it that way. The core messaging features of celery are still available for the developers to use. The JSON-RPC interface is another optional interface to using our API. This is more for interacting with additional web services and communicating with our other offices. The "direct" api interface could also be configured to work with our other offices if necessary. It's basically just changing the RabbitMQ broker url :). I do see how writing code to appear as if it's running locally could lead to confusion or ignorance as to what is actually happening. On Sat, May 31, 2014 at 5:22 AM, Marcus Ottosson <[email protected]> wrote: > you don’t think that retrieving or querying the status from a remote > backend for the status of a distributed task would be performant? > > I think it’d be about one billion times slower than querying anything > local. :) > > What I’m referring to though is the design aspect of RPC. I’ve been > reading some rather discouraging information about it lately (this one > <http://steve.vinoski.net/blog/2008/07/01/convenience-over-correctness/> > sums up it up rather well, and this one > <http://www.inspirel.com/articles/RPC_vs_Messaging.html> goes through > their differences) and have been staying clear for the sake of finding out > exactly what can be gained by doing something else; in this case - > messaging. > > The argument basically boils down to the fact that making a local call is > faster (by the billions) than making a remote one, and that dressing a > remote call up to look local encourages bad design. I’ll try and > illustrate, although I’m still looking to find exactly what those pros and > cons are: > > import studiox > def rpc_publish(asset): > """Example of RPC hiding slow calls. Which are local and which are > remote?""" > path = studiox.publisher.get_path(asset) > variant = studiox.path.dirname(path) > > # Perform quality checks > assert studiox.qna(variant) > assert not studiox.islocked(asset) > > > studiox.commit(asset) > studiox.push(asset) > > # Notify subscribers (database, peers) > studiox.publish(asset) > > Compared to a message-based one, where each function - or “service” - is > de-coupled, including handling of errors and distributed logging: > > import studiox > def soa_publish(asset): > path = studiox.publisher.get_path(asset) # Local > > studiox.messaging.Request(service='dirname', payload=path).send() # > Remote > variant = studiox.messaging.recv() # Blocking > > studiox.messaging.Push(service='asset.qna', > payload=asset, > reply_to='islocked').send() # Asynchronous > def soa_islocked(asset): > result = studiox.islocked(asset) > if result is not None: > studiox.messaging.Push(service='asset.commit', > payload=asset, > reply_to='push').send() > else: > studiox.messaging.Push(service='asset.error', > payload='%s is locked' % asset).send() > def soa_commit(asset): > studiox.commit(asset) > studiox.push(asset) > > studiox.messaging.Publish(service='log.published', > payload=asset).send() > > Clearly more verbose, and this is where I suspect convenience may > influence a design, potentially for the worse. > > how do you mange and monitor all of your celery workers? > > At first, this question stuck me as odd. But from what I gather, RabbitMQ > acts as a broker, in which case you’re relying on existing implementation > for features such as logging and monitoring. > > Ultimately, RabbitMQ (and others) are higher-level than ZeroMQ and in this > particular example swarm.py is playing the role of RabbitMQ’s “server” > application. > > So, the reason I found it odd was that, having written swarm.py, logging > is merely an additional call from the broker to another worker; a logging > worker. Monitoring it yet another call and so forth. At this point, both of > those are rudimentary and aligns with existing functionality. > > “broker” of swarm.py > <https://github.com/abstractfactory/labs/blob/ef788903194423c851337f49c8d202f6cb745e1f/python/messaging/chat/swarm.py#L57> > > Simple, unless there’s something I’m missing. > > > -- > You received this message because you are subscribed to the Google Groups > "Python Programming for Autodesk Maya" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOCek-bmWcwNt%3D%2BQnT%2BYAn27G%2BVJ6Qv2UUOExQh%2BF4SaHQ%40mail.gmail.com > <https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOCek-bmWcwNt%3D%2BQnT%2BYAn27G%2BVJ6Qv2UUOExQh%2BF4SaHQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- -tony -- You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAJhmvsSfQ%3DRLkmOT6gYnMNcwghQB8nOQZUhkKRHP6e9hS1Gqzg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
