I'd be pretty surprised if this caching scheme is faster than running queries into MySQL, that sucker is BRUTALLY fast! If you build derived tables and use those as caches (so you just basically are doing "SELECT * FROM table") it will almost certainly exceed the speed at which you can deserialize the data with storable. For these sorts of queries where you are only reading and doing none or simple indexing its generally about 100 times faster than Oracle, and 10 to 50 times faster than postgres. This is especially true with large tables. I have a customer with 2 tables of close to a terabyte of data in them and upwards of 100 million rows. SELECT times into these tables on a dual processor PIV Xeon with 2 gigs of ram and a ciprico RAID 5 box are routinely on the order of sub 1 second.
On Friday 01 February 2002 09:34, Mike Chamberlain wrote: > > "Mike Chamberlain" <[EMAIL PROTECTED]> writes: > > > We use a multiple stage pipeline to simplify the WML creation > > > at various stages. We don't use the internal AxKit cache > > > system, due to us providing customised pages based on postcodes > > > (that's 1.7 million variations of each page). We do however > > > have our own SQL caching layer. > > > > Just out of interest, how does your SQL caching layer work? > > OK, our site architecture is based around publishing fairly static > data.[1] When we roll out data to our front end database servers, > we touch an update file on each of the our web servers. > > When we do a sql query, we check a cache of storable objects > for the existance of a results cache file, if one exists we > parse the query to find out what tables it uses, and from that > list we work out the latest update time for any of the tables > involved in the query. If any of the tables are newer than > the object from the cache, the cache is invalid. > > Otherwise, if no cache objects exist, do the query and turn that > into a datastructure which can be serialised out to disk using > Storable. > > Cache keys are worked out as md5 hashes of the sql query. > > Queries can be set as cacheable, or not cacheable if the > data is truely dynamic. > > We also have mechanisms to turn off caching for a http request/session > regardless of the query so that we can debug any problems with the > databases. > > [1] Eg, house price data is updated monthly, postcode databases quarterly, > classifieds serveral times a day. > > Mike. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
