On Wed, Jun 19, 2013 at 12:33:44PM -0700, Suhrawardi wrote: > > (...) > > > In our case, we would like to start a lot of workflows at the same time > > > (100 / 200) and have them run all at the same time (as much as possible) > > > and return asap. We do a lot of IO stuff, databases / http calls etc, so > > > that should speed things up. > > > But I have a feeling that when I start 200 workflows with the example > > poc > > > application, the first step is started 200 times. When that first step > > is > > > finished for all of them, it looks like it is waiting for 10 seconds, > > and > > > only then continues with the second step. > > > > Yes, ruote 2.0 version will do that exactly, it will do the 200 first > > steps > > and then the next steps will all quick in. If there are multiple workers > > (and > > slow/human participants afterwards, that changes...) > > > > So, if I understand you correctly: When I start 50 workflow processes at > the same time, for all of them the first participant is handled. When all > replied, only then the second participant is handled? That means that if we > have one of these 50 workflows that takes a long time (eg gets a timeout > because the required resource is unavailable), this will block all the 50 > other workflow processes? > > Hmm, in our case that is not an option... :-) We'll have to avoid that. But > using different engines, that should not be a problem.
Hello, sorry, I didn't explain it right. (speaking of ruote 2.x) When you start 50 workflow processes together, the first node of their expression tree is handled, 50 in a row. As soon as a participant node is hit, a workitem is dispatched to the participant, when the reply comes back to the engine, the reply is scheduled for execution. The workers pick the messages as they come, so when a participant takes some time, its answer is delayed and work for other processes get executed beforehand. Ruote workers are very dumb, they just pick up available work and perform. What you see is 50 messages getting processed one after the other. If you look carefully at the output of the workers from my ruote_poc, you'll notice that even if it looks like a pack of 50 sheeps advancing on one front at the beginning, it then trickles and then some processes are processed faster. The participant in the ruote_poc are very simple, but they execute each in its own Ruby thread (this default behaviour can be changed), so as the executions move on, the processes whose successive participants executed faster (the whim of the thread scheduler) finish faster. It can be thought to simulate somehow participants calling out to external services. Branches of workflows calling out to participants place themselves in wait for the participant answer. That waiting time may vary. For ruote 3.0, as I did for some storages and was demanded/suggested by some people in this list, I'd like to try to keep a worker processing the same workflow instance until it hits a participant. At that point it'd schedule the participant work and yield, probably considering other workflow instances. That would alter the "50 sheeps walking on single front" effect you've been observing *right after the launch*. Wow, that is long. If you want to observe the work in each worker of my ruote_poc: https://github.com/jmettraux/ruote_poc/commit/d822e5a65f39 > (...) All the best, kind regards, -- John Mettraux - http://lambda.io/jmettraux -- -- you received this message because you are subscribed to the "ruote users" group. to post : send email to [email protected] to unsubscribe : send email to [email protected] more options : http://groups.google.com/group/openwferu-users?hl=en --- You received this message because you are subscribed to the Google Groups "ruote" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
