Hey John, et al., I had a problem where calls to the Ruote::StorageHistory were bloating our Ruby process's memory footprint by ~370MB, bringing our servers to screeching halts. Upon investigation, I found that Ruote::StorageHistory#by_process pulls all the documents from the history database and iterates over them. While it might only return 100 documents, it causes the Ruby interpreter to grab enough memory for all 16.5k records (and the Ruby interpreter doesn't like to let go of memory it's gotten its greedy little hands on).
My first attempt at a fix was to paginate the fetching of the records. This keeps the memory footprint constant as the database grows, but adds significant I/O time. It's a good fallback because I prefer that things take a long time, rather than crashing. My second attempt was to make StorageHistory leverage WfidIndexedDatabase, and it was actually much easier to implement. It reduced the memory footprint while also increasing performance by a factor of a little less than 4. Here's the gist with my monkey patches: http://gist.github.com/570681. I thought some portion of them might be suitable for incorporation into mainline. I haven't done extensive testing or field-use of these yet--that will be coming in the next few weeks. I'll keep the gists and the list updated with any significant patches. -Ian -- you received this message because you are subscribed to the "ruote users" group. to post : send email to [email protected] to unsubscribe : send email to [email protected] more options : http://groups.google.com/group/openwferu-users?hl=en
