On Jan 31, 2009, at 3:37 PM, John L. Singleton wrote:
> > In addition to what Jacob said, here's my two cents. :) > > Performance isn't the only reason to consider clustering. The other > reason is availability. If you have an application that must > absolutely be available 24/7 then you must cluster. IMHO there isn't > really a way around this if availability is a requirement. The > thinking is that it's highly possible that *one* machine can > experience some fatal hardware failure but much less likely that *all* > of your machines will... at least at the same time. Absolutely true. I didn't mention it because if you need that level of reliability, you've probably started to get into redundant servers and clustering anyways. It all comes back to cost. Adding uptime beyond the typical 95% uptime increases costs exponentially. > If there's any possibility you will have to cluster your app, make > sure to code it in a way that *will* cluster. Coding an application to > be distributed/parallel is a little different than the assumption that > it will be always running on the same machine. What does that mean? Before you do this, I'd recommend taking a realistic look at your app and ask if this will _ever_ be a problem. For most websites, speed of iteration and minimizing developer expenses are far more important than the ability to scale out. Write good modular code with unit tests and you won't many problems modifying it to scale out when you have to. > > Well, it can be simple bone-head things like *assuming* that uploaded > files exist much later, or even the assumption that a users' session > will only exit on one machine per session. When you cluster, all of > this goes out the window. Every box has to be ready to pick up the > session, wherever it left off. Things like sticky session help with > this but you don't want your failover strategy to require a user to > close their browser and try again. :) So think about this when you > code your app. You don't want to be hitting a performance wall, > getting ready to cluster, and finding out that there are hundreds of > "this-will-only-run-on-one-box" assumptions. Then you get into the > territory Jacob was talking about. You will run into those problems regardless. There is always something that doesn't work the first time out. Or the second or third. These problems become readily apparent about 30 seconds after you add your second machine. Or at 1am Sunday morning 3 days after a publish. Race conditions happen. You will start deadlocking your database. You will create too many database connections. Aggregating and analyzing server logs becomes a chore. Your memcache server will fail and load will spike on your db causing your site to become unresponsive. And on and on and on. Little blips and spikes that you didn't think twice about when you were processing 100k pages/month will absolutely kill you when you're doing 10M+ pages/month. > If you do cluster: > > *Build servers from images, not fresh installs. > *Figure out ahead how you're going to manage your deployment. (My > sfSyncCluster Plugin is good for this :) > http://www.symfony-project.org/plugins/sfSyncClusterPlugin > ) > *If you want to go with a software load balancer, try HAProxy: > http://haproxy.1wt.eu/ > --- Few hosts offer load balancing, so if you host anywhere you'll > have to build your own cluster. I've used Linux Virtual Server with great success as well. It's a bit annoying to setup but it works quite well with little overhead. > > *Work out your failover strategy ahead of time and *test it* --- it > all sounds good in theory until you try it and it doesn't work. Test > "emergency" situations and time how fast you recover from them. Absolutely. I cannot stress this enough. You must test anything that can fail and you must test it under load. Unplug a server under full load, wipe its drive and see how long it takes you to put it back into the cluster in 100% working condition. Make sure you have n+1 webservers - you will need to take one down at some time and you need to be able to handle the load. Keep spare parts on hand - ram, hard drives, cpus, fans, PSUs. I also cannot stress enough that you MUST have backups and they MUST work. All of this redundancy and capability is worthless if you get hacked and he deletes all of your files. Or if the data center is hit by lightning. Bad things happen, be a pessimist and be ready for them. You also MUST plan for PEAK (or very-near-peak) usage not average. If you plan capacity for average usage, you will never hit peak and you're shooting yourself in the foot. > > > That's it for now... > > Best, > JLS > > > On Jan 31, 2009, at 7:43 AM, Jacob Coby wrote: > >> >> >> On Jan 31, 2009, at 5:28 AM, Sumedh wrote: >> >>> >>> Very good discussion... :) >>> >>> Idea of creating a Ramdisk and putting cache folder on that is >>> neat... :) >> >> Yes, neat. And totally worthless on a Linux server. Linux already >> uses excess ram to cache commonly accessed files. You're just taking >> ram away from processes that could actually use it. A good portion >> of >> it will be kept empty since you have to allocate more space than you >> actually use. I imagine the same applies to *BSD and probably >> Windows >> as well. >> >> Ramdisks were a great idea around MS DOS 5 when you wanted to get rid >> of diskette seek times. They're worthless now. Use a php opcode >> cache instead. You'll get a 10000x better ROI on that 64mb than >> adding a ramdisk. >> >>> >>> >>> Can someone point me to a resource that tells me typically what is >>> the >>> amount of traffic when one needs to think of clustering very >>> seriously... >> >> You look at clustering when you can't operate on just one server >> anymore. There is no typical amount of traffic. It's wholly >> dependent on your site's performance and your (or your company's) >> willingness to spend money. >> >> In short, you will know when you need to start looking at it because >> it will be your only option. You've added an opcode cache. You've >> added a dedicated database box. And then upgraded it. You've >> upgraded your webserver. You've added memcache. You've enabled sf >> caching. You've optimized slow queries. You've triple-checked the >> database settings. You've gone over the code a dozen times to make >> sure you're not using a boneheaded algorithm somewhere important. >> You >> still can't keep up with traffic. It's time to start looking at >> scaling out. >> >> It's been my experience that you should avoid it as long as possible. >> Adding multiple servers that perform the same task will lead to >> headaches and rising expenses and will slow down development. >> >>> >>> >>> And which are good hosting providers offering load balancing etc? >>> >>> On Jan 28, 6:12 am, Pablo Godel <pgo...@gmail.com> wrote: >>>> I was not using symfont at the site. I was trying to cache queries >>>> results and rendered templates. >>>> >>>> Pablo >>>> >>>> On Tue, Jan 27, 2009 at 7:46 PM, Lee Bolding <l...@leesbian.net> >>>> wrote: >>>> >>>>> Memcache can cache anything - be that a DB query, an object, >>>>> etc... >>>> >>>>> http://code.google.com/p/memcached/wiki/FAQ#Cache_things_other_than_S >>>>> ... >>>>> ! >>>> >>>>> On 28 Jan 2009, at 00:34, Eno wrote: >>>> >>>>>> On Tue, 27 Jan 2009, Pablo Godel wrote: >>>> >>>>>>> I rememeber trying this and I did not see a noticeable >>>>>>> improvement, I >>>>>>> am not sure if I did something wrong. Instead, we used memcached >>>>>>> and >>>>>>> we did see a big difference. >>>> >>>>>> memcached caches your database queries, but maybe your actions >>>>>> were >>>>>> not >>>>>> fully cached? On a site with a lot of pages you will have a huge >>>>>> cache (in >>>>>> our case, a couple Gb of files!). >>>> >>>>>> -- >>>> >> >> -- >> Jacob Coby >> >> >> >> >> >> >> >>> > > > > -- Jacob Coby --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "symfony users" group. To post to this group, send email to symfony-users@googlegroups.com To unsubscribe from this group, send email to symfony-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/symfony-users?hl=en -~----------~----~----~----~------~----~------~--~---
