On Mon, Oct 31, 2011 at 10:23 PM, Paul Graydon <[email protected]> wrote: > A few quick thoughts, and a few useful resources.
I think these are all great suggestions, but I think there are even more basic things startups can do to help scalability. These are my pet peeves of web apps and startups that make the first page of Hacker News and fall over. * Don't host on GoDaddy shared hosting or similar services. A lot of Platform Clouds (Heroku, App Engine, OpenShift) are free for a basic level. So you can develop for free and pay more as you scale. If the company doesn't have any sysadmins/ops, they are probably better off on a PaaS. * To scale at all your app needs to be able to be load balanced. Some load balancers have stick sessions, but a lot don't. So your app servers should not keep local state, that means session cookies need to be stored in a database not on the local file system. For sessions use a simple fast DB like Redis, or cache with Memcache. User uploads should ago to shared storage like S3. * Eventually that large monolithic Rails app will need to be broken up, e.g., backend, frontend, search, live feed, etc. If you can easily make it more modular now that will help you if you ever get big. But don't waste too much time on it, you can't predict the future, so get your Minimum Viable Product out the door. Here are some good suggestions on scaling Rails: http://axonflux.com/building-and-scaling-a-startup -Anton > Develop against the technology you're going to use in production. I know > that seems absurd to have to say, but it's amazing the number of apps you > see out there that are tuned for things like Microsoft Jet or SQL Express > which the dev was running on their local machine. If you're going to run it > on MySQL in production, develop against it. Different databases handle > queries in different ways, what is fast on one isn't necessarily fast on the > other, and for pities sake make sure it's the same version too. MySQL 5.5 > performs a lot better than MySQL 5.1 for example, and 5.6 is faster than > 5.5. Even down to the minor version is good, 5.5.12 (IIRC) had an > interesting performance bug that bit us. > Also aim to develop on data of at least reasonable scale. It's nice your > function runs quick against 10,000 rows in your dev database, but what if > it's typically running against a million in production? I was working with > a dev who had a stored function that ran in <1s against the dev database but > would typically take over a minute and a half against production in real > world scenarios. > Write your apps with split master/slave SQL from the outset, even if you're > pointing them at a single database to start with. That way it's a lot > easier to bring in master/slave down the line, rather than having to go > through a bunch of code later on to point them at appropriate servers. Even > if you end up with a database structure that doesn't use master/slave the > amount of work required at the offset is fairly minimal. It's a bit of an > on-going disagreement I have with a few of our developers. Should you find > yourself in a position to need to rapidly scale it'll be a lot easier to do > so and buy yourself time for further considerations on how to best scale > your app (i.e. would sharding be preferable?) > Cache, cache, cache, cache, cache. No point wasting CPU/Database time > answering frequent queries. Cache wherever possible, as close to the edge > as feasible, but be conscious of security. You don't want to cache secure > content and return it to the wrong person. Stick a memcache layer in or > similar. > Make static whatever can be. No point generating the entire page from code > if the content is staying static for hours at a time. > Use config management. Puppet / Chef / Cfengine / Whatever. If you're > lucky enough to have your site go viral you're going to need to scale very > quickly and you don't want to have to be building and configuring every > server from scratch when config management can do it for you in minutes. > On that front, if you're running your own physical servers don't forget the > cloud can provide you temporary breathing space. A former colleague of mine > works for a start-up that much prefers physical hardware. Their monitoring > system tracks overall site usage & performance metrics. Should they start > to reach a defined threshold the monitoring system will automatically > trigger provisioning of a new virtual server, and tells the config > management about it so that it can be set-up and added into the loop without > a single finger being lifted. Tech team are e-mailed at the same time so > they can investigate if necessary, and if it happens frequently enough they > can consider buying more hardware. > http://techblog.netflix.com/2010/12/5-lessons-weve-learned-using-aws.html > Chaos Monkey. Build your platform to work as best as possible even if > certain services are down. _______________________________________________ Tech mailing list [email protected] https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
