On Sat, 2007-04-28 at 08:07 -0400, Dan wrote: > IMHO. I never implemented memcache, but the distributed memory > concept is *not* by itself scalable architecture. The key is session > data. Typically (as in default Pylons setup) session data is tied to > physical hardware... your web server. To make use of memcache, one > would first need to implement scalable architecture, such as the > 'share nothing' approach.
And this is what Memcached does. Simply store your session data in Memcached. > > Again, I'm not claiming to be an expert on the 'share nothing' > architecture, but my understanding is that 'share nothing' stores all > data in the database... even session data. But of course, "database" doesn't have to mean "SQL database". Memcached provides a "hash in the sky" quite ideal for this type of problem. > This way, it doesn't matter what web server in your farm takes users > requests. They will all ask the database (the db will be your > bottleneck) but at least they will *mostly* know how to access and > store data. Again, this is exactly what Memcached is meant to do. Memcached can share data transparently across servers and will typically be *much* faster than database access (and much, *much* easier to implement - distributed databases are not fun [or cheap]). If you'll recall, Memcached was developed to help solve LiveJournal's scalability issues, so it's quite well-tested: http://www.linuxjournal.com/article/7451 Also, if your architecture consists of storing your data in a single database, you will certainly *not* scale beyond a certain point. The database itself will become a bottleneck. In fact, if you read the above article, you'll find that Memcached was in fact developed to help remove load from the LiveJournal databases. > So the basic point to my post is that scalable architecture is more > about theory than specific tools. The theory behind building > scalable web applications is a growing subject in system engineering > that I find interesting. You can check a decent article about > myspace.com struggles at - > http://www.baselinemag.com/article2/0,1540,2084131,00.asp > I found the articles on scaling Wikipedia interesting, although I can't find the link to them at the moment... Despite the fact that MySpace obviously scales to a large degree, I put little trust in their technical abilities (they can't even do CSS or search properly). I suspect their scalability is largely derived from millions of dollars in investments rather than actual architecture. *Anyone* can scale if they can throw unlimited money at it ;-) > 'memcache' is a tool, when applied to a scalable architecture, could > provide performance improvements. Distributed information is a cornerstone of a scalable architecture (sessions are simply information in this context). Memcached provides exactly this and is therefore more than just a "tool". It's a foundation point for building a scalable architecture. Of course, for most of us, this is interesting but purely theoretical. For most of the sites I do, the required scalability of a Pylons app consists of running multiple instances of the app behind a load balancer on the same server ;-) Of course, the same problems and solutions apply but on a much smaller scale. Planned right, scaling from multiple processes on a single machine to hundreds of processes across multiple machines really is a matter of scale rather than architecture. Regards, Cliff --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
