I have a similar problem with my blogging software. Most of the content is static, and it's kind of a waste to regenerate everything every time someone views the page. My solution to this problem is to use a a Cache::FastMmap object to store the formatted HTML content of each post, keyed on the source material's MD5sum (and filetype, for reasons not relevant to this discussion :)
This results in a nice speedup, since certain HTML filters are slow (OO.org document -> HTML, for example). I also cache full pages, after they've been assembled by TT, keyed on the md5sum of each included article and some metadata (comment count, tags, etc.) The end result is a 5-20x speed up. With Catalyst's embedded server, I can do about 5 requests per second without cache, but it goes up to about 40-50 requests/second with the cache. So obviously a speedup. (And I'm still not doing a very good job, this was just a 20 minute hack!) For the gory details, check out the code here: http://www.jrock.us/trac/blog_software/browser/lib/Blog/Controller/Root.pm#L105 (page cache) and: http://www.jrock.us/trac/blog_software/browser/lib/Blog/Model/Filesystem/Item.pm#L355 (object cache) Another caching strategy is to set dynamic pages to be cachable for say, 10 seconds, and then letting Apache's mod_cache (experimental in 2.2.3, but it works for me) do the hard work. Your application gets hit once every ten seconds to refresh Apache's cache, but all the end-user requests are handled by serving from Apache's memory cache. With mpm_worker, it's *super fast*, and a 10 second update delay isn't bad. If your application is really slow, this will let you increase your slashdot resistance factor (as I call it) enormously. A quick ab against some cached content gave me a requests/per second rate of 1638... much better than whatever your app would do by itself. And you don't really lose much by taking this approach, other than a 10 second delay with your articles going live... but 10 seconds is nothing if you're getting 1500 requests per second and handling it with an old PIII :) If this info is useful to anyone else, let me know (off-list) and I'll write up a more detailed article about it :) Regards, Jonathan Rockway > Thank you, you raise a good point that I had not fully considered. My > goal is to display a dynamically generated page, but where most of the > "dynamic" content is actually static for a period amount of time. (Take > a news page, for example, where the list of news stories updates once an > hour.) > > Having thought about the problem a bit more, I can likely simulate what > I'm looking for by using a template that INSERTs the "dynamic" content > that was generated by a cronjob periodically. This would move the heavy > DB processing offline relative to the page view. > > Randal, thank you for your insight! > > Have a great night. > -- Robert > >
signature.asc
Description: OpenPGP digital signature
