On Mon, Jun 17, 2013 at 06:11:16AM -0700, Suhrawardi wrote: > Hi, Hello Jarra,
welcome to the ruote mailing list. Thanks for the excellent analysis work and issue report, very much appreciated. Sorry for the problems. > We are building a workflow system for a process that queries some other > sytems (DB's and via http) and use ruote-redis for storage. When I start a > single workflow process, the whole process is finished within 10 seconds. > But when starting more processes (15) at the same time, the processing time > increases to 30 seconds. When we start even more, we see that eventually > all the workflow processes get stuck. > > To make sure that the database or http calls are not the problem, I removed > all these calls, and replaced the code in the :on_workitem calls with a > dummy code that adds a dummy value to the workitem and does a reply. That > speeds up the workflow process very little to 20 seconds, while a single > workflow at a time with db and http calls is finished in 10 seconds. Well done. > This seems similar to the problem described in > this https://groups.google.com/forum/#!topic/openwferu-users/0shOiU7ab8M > post. > > We didn't manage to speed up the process with more instances of the > workflow application (we use AMQP to send the messages, so we can easily > add more instances), or by running multiple workers of engines within a > single instance of the workflow application. Multiple worker threads, one GIL. Independent Ruby processes workers should be better. > Switching to the FS Storage didn't solve the problem either. Do you mean that when launching simulaneously 20 ruote processes (one ruote worker), the system jams (Redis and Fs too) ? > So far, we didn't have any luck solving this problem. Do you have any ideas > what the best approach would be to pinpoint this problem? > Any ideas where to look? > > We use ruby-2.0.0-p195, ruote (2.3.0.2), ruote-redis (2.3.0). If I understand correcly, it's purely ruote's fault. May I see what the process looks like? I'd like to try myself with the Redis storage and the fs storage. I tend to run with noisy on, and/or, since it's Ruby code, I'd do "bundle open ruote" and add debug output to the #step method in the lib/ruote/worker to see what is happening. I'd really like to see the process to see what kind of stress (indenpendtly of participant implementations) is put on the ruote worker and the storages. Thanks in advance, sorry for the nuisance, -- John Mettraux - http://lambda.io/jmettraux -- -- you received this message because you are subscribed to the "ruote users" group. to post : send email to [email protected] to unsubscribe : send email to [email protected] more options : http://groups.google.com/group/openwferu-users?hl=en --- You received this message because you are subscribed to the Google Groups "ruote" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
