On Fri, May 15, 2009 at 3:02 PM, choda <[email protected]> wrote: > > Dear ruotists ;)
Hi Marc I'm keeping trying to use ruote wisely for the project i'm working on, > and a few technical questions have arisen. > > Basically, what I need is to dispatch the processing of numerous files > between several resources, each of which is not necessarily available > at a given time. This for me is the crux of your entire mail. I'm going to do my best to answer everything, but I'm going to summarize here quickly. You're effectively looking for a way to distribute large number of files (or "work") to a variable number processes. These processes will perform said tasks upon receiving "work" and idle around waiting for more. You want each process to quickly pick up the next available piece of "work" to prevent being wasteful. Over and above this, you're looking at ruote as a way to orchestrate all of this, so it happens in an orderly fashion. How right/wrong am I? I deleted the rest of your mail on purpose, I feel the lines between the workflow engine and the actual bodies of work you're describing is blurred too much. It happened to me to when I started using ruote, so don't worry. First off. Are you sure ruote is the right tool for the job? Unless I missed something obvious (very possible) it appears to me that you want to use ruote as a delegation/communication mechanism, and not as a business process engine. There is, at least in my honest opinion, a distinct difference here, and it will make you love ruote or hate it. More at the end of the mail... Second off. Why isn't a message queue like AMQP sufficient? AMQP is an awesome message queueing and routing protocol. I'm using it extensively in our systems with ruote. I'm cleaning up an AMQP participant/listener pair that I'll pass to John for merging over the weekend. How would I do this? I'd start using a pure AMQP publisher/consumer model, without ruote (sorry John). The flow might look something like this: = Publisher = 1. File accepted 2. Prepare "work" (JSON) and push into AMQP durable exchange 3. Listen on another durable queue for feedback 4. React to feedback 5. Rinse and repeat = Consumer = 1. Subscribe to a message queue 2. Receive "work" (JSON) 3. Process instructions 4. Send feedback to publisher I've reduced the problem significantly, but hopefully makes it more clear. Ruote will only benefit you on the publisher side when your process needs more than "if respone == true; job_succeeded; else job_failed; end". Ruote strengths (for me) lies in the ability to react on the responses. Failures change the entire course of the process, people get notified, actions have to be taken. Ruote isn't the right tool to use for "watching a directory" and "starting processes". It is a decision maker, it calls the shots. I can ramble on here until I need an ISBN for the reply, so I'm gonna stop. Please give us some more info so we can help you disect the issue even further. Sounds like an interesting problem you have to solve, curious to learn more. Best -- Kenneth Kalmer [email protected] http://opensourcery.co.za @kennethkalmer --~--~---------~--~----~------------~-------~--~----~ you received this message because you are subscribed to the "ruote users" group. to post : send email to [email protected] to unsubscribe : send email to [email protected] more options : http://groups.google.com/group/openwferu-users?hl=en -~----------~----~----~----~------~----~------~--~---
