Hi John, Thanks a lot for your reply.
> Because we use AMQP, we can easily start more then one process > (bin/app.rb) > > to handle the workflow requests that come in. The load will then be > divided > > by the different processes that are running. That's what I mean with > > multiple processes. > > So you want to distribute the work among "engines" and not among "workers > of > a single engine". > I think that distributing the work among different engines should perform better in our case. > In our case, we would like to start a lot of workflows at the same time > > (100 / 200) and have them run all at the same time (as much as possible) > > and return asap. We do a lot of IO stuff, databases / http calls etc, so > > that should speed things up. > > But I have a feeling that when I start 200 workflows with the example > poc > > application, the first step is started 200 times. When that first step > is > > finished for all of them, it looks like it is waiting for 10 seconds, > and > > only then continues with the second step. > > Yes, ruote 2.0 version will do that exactly, it will do the 200 first > steps > and then the next steps will all quick in. If there are multiple workers > (and > slow/human participants afterwards, that changes...) > So, if I understand you correctly: When I start 50 workflow processes at the same time, for all of them the first participant is handled. When all replied, only then the second participant is handled? That means that if we have one of these 50 workflows that takes a long time (eg gets a timeout because the required resource is unavailable), this will block all the 50 other workflow processes? Hmm, in our case that is not an option... :-) We'll have to avoid that. But using different engines, that should not be a problem. > Here is my test bench: > > https://github.com/jmettraux/ruote_poc > > Here are the stats for one set of runs: > > https://github.com/jmettraux/ruote_poc/blob/master/results/stats.txt > Thanks a lot for your take on our poc. It is very insightful and helpful. We saw you used Yajl. When we changed to Yajl, that halved the processing time... :-) At no point does the engine jam. I see no "steps start multiple times and > doesn't finish" issues. > We're investigating further. And it looks like it's caused by Bunny losing messages under high load. So, still some more investigation to do. But you helped us a lot! Thanks a lot! Jarra -- -- you received this message because you are subscribed to the "ruote users" group. to post : send email to [email protected] to unsubscribe : send email to [email protected] more options : http://groups.google.com/group/openwferu-users?hl=en --- You received this message because you are subscribed to the Google Groups "ruote" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
