Awesome work. I removed all my elaborate locking, which I had been blindly trying to copy from the Redis adapter, and used your pure optimistic concurrency approach instead for a noticeable speed boost.
I finished my proof of concept on an optimized storage implementation and was partially successful. My approach was to create a storage that would fully execute an incoming message and any messages resulting from the application of that message persisting to in memory storage in a sort of transaction or "unit of work" and then persist to the DB at the end of the transaction. Concurrency is handled by locking on the WFID rather than the expression id, and a transaction is processed without stopping until it is complete. "Complete" means that if the expression initially processed caused any other messages to be placed on the queue, those are all processed as well, recursively, until no more messages are generated (i.e. no more expressions will reply). This approach was effective, but not dramatically so. My load test launches 10 simultaneous workflows, replies a work item, and waits for five more items from a concurrence. I was able to improve the performance of this environment of the Mongo storage by about 50%, and the Redis storage by about 30%. These percentages were consistent (even a little better) when the load was 100 simultaneous workflows instead of ten. One interesting side effect of this work is that the characteristics of Ruote's response to load are different. Originally, due to the strictly round robin nature of putting fine grained, atomic components of a single logical method call into a fifo queue, all work would always be completed at roughly the same time. If the load was high, the work would take a long time to complete, but complete pretty much all at once. With the alternative techniques, work is completed in a first come first serve fashion, and the workflows tended to complete one at a time throughout the life of the test. I was expecting more striking results, but the result is good enough for our operations. This technique will allow us to comfortably stay with Mongo DB for quite some time, keeping Redis in reserve if we need another boost. Redis processed 10 workflows using 2 workers in about 10 seconds using my hybrid storage, and Mongo DB took an average of 14 seconds. By contrast, without the hybrid storage Redis averaged 15 seconds and Mongo DB 30 seconds). All mongo tests used your improved approach to concurrency. The experiment wasn't effective enough that I intend to package this stuff up or make it generally available, but if anyone else is ever in our shoes I wouldn't mind doing so upon request. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of John Mettraux Sent: Wednesday, November 23, 2011 12:53 AM To: [email protected] Subject: Re: [ruote:3311] Re: Change storage implementations in production and other questions :) On Mon, Nov 21, 2011 at 03:06:41PM -0800, Nathan Stults wrote: > > On Fri, Nov 18, 2011 at 04:56:39PM -0800, Nathan wrote: > > > > As for the mongo storage, I would love to go through the code with > > you. I'm sure it could be made much better. I'm not sure how we > > would do it, but I'm very open to it any time you like. > > You don't mind if I give a try at it from scratch ? (So I can learn > MongoDB as I go). Hello Nathan, here's my take on the ruote + MongoDB: https://github.com/jmettraux/ruote-mon It's using :safe => true and all the Mongoodness available. It's multi-worker safe. I wanted to have an idea of its relative speed, so I ran the whole functional test suite of the latest ruote (b66b34f) on Ruby 1.9.2p290. 2.4 GHz Intel Core 2 Duo, with 4GB (Macbook 2008), (3 tries each). Ruote::HashStorage 303 / 300 / 302 seconds Ruote::FsStorage 362 / 363 / 353 s Ruote::Redis::Storage 388 / 389 / 389 s (redis 2.2.12) Ruote::Mon::Storage 410 / 413 / 413 s (mongod 2.0.1) Ruote::Sequel::Storage 468 / 469 / 477 s (sequel 3.20.0 + pg 8.4.2) I don't know how it would fare in your test harness. I'm sorry, I have no result for ruote-mongodb, I've tried to run it (current master) two or three times, each time it got stuck after five or six tests. The code is minimal, I did some tiny adaptations to the ruote test/unit/storage.rb to accomodate for BSON hashes, but not much more. Maybe the #put method can be simplified. I didn't know MongoDB was so pleasant to work with. What do you think ? -- John Mettraux - http://lambda.io/processi -- you received this message because you are subscribed to the "ruote users" group. to post : send email to [email protected] to unsubscribe : send email to [email protected] more options : http://groups.google.com/group/openwferu-users?hl=en -- you received this message because you are subscribed to the "ruote users" group. to post : send email to [email protected] to unsubscribe : send email to [email protected] more options : http://groups.google.com/group/openwferu-users?hl=en
