Re: Building high load mod_perl/Mason servers
General ideas are in Stas' introduction and other mod_perl books. Here are some practical numbers which may be useful for your reference. (assuming that all your servers have 1G RAM) 1) when daily unique IP are about 25K. Run mod_perl on the database machine with persistent Apache::DBI database connection. Turn one of the two frontend servers to be a light Apache server that a) serves all static contents and b) proxy to the mod_perl server for dynamical content. Leave the other frontend server to serve temporary PHP programs. With 1G in the frontend server, you are okay to run 500 MaxClients. 2) daily unique IPs are about 50k. Run both the frontend servers to be light Apaches and proxy to the mod_perl for dynamic contents. Memory may just be used up in the DB/mod_perl machine. If it is going to be a problem, try to remove Apache::DBI and use database cache to save memory but still keep a fast DB connection. Also, design the programs carefully to use caching ability in the light servers and/or to return proper headers (e.g. NOT_MODIFIED) as soon as possible. 3) daily unique IPs are about 100k. May need 3-4 frontend light Apaches, 1-2 mod_perl servers and 1 DB. Only with a daily unique IPs above 100k, one will defeintely needs two or more mod_perl servers. Synchronizing mod_perl codes should not be a problem --- for examply, simply mounted as NFS. Mod_perl calls the codes only once when starts. 4) Mason and other tools --- one can take the advantage for general development purposes. If the site has only a few specific services, it is better to write mod_perl from scratch than with a tool. These numebrs changed from sites to sites, I believe. The above numbers are based on our experience only. A popular web site usually contains many clients of slow network connections. So proxy is the key to serve them efficiently. Peter Bi - Original Message - From: Eric Frazier [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, June 06, 2002 7:43 PM Subject: Building high load mod_perl/Mason servers Hi, I just got the chance to be in charge of a very busy website, and its future growth. Currently it is running with PHP on two round robin servers with a separate mysql db server. We are going to be moving to mod_perl, but I am worried about how to keep from getting into the same kind of trap with mod_perl as with PHP. The PHP guys don't know OOP, so we have to code halfway, modules exporter but not OOP modules. It has to be something OOP like without getting too complex at first. The PHP trap is just the horrible require once stuff all over the place and global vars etc. I know lots of people blame this kind of coding on perl geeks, but the PHP stuff I have been seeing is pretty bad with it because the constant thought is must fix current problem wait till later to be pretty but later never comes. Also things like using ten instr functions instead of one reg exp. So I am thinking whatever I do it should fit within an existing framework, something like Mason. But I am confused about what real advatage Mason provides, and how things like source code control would work if we are running lots of servers. Do people use rsync to keep up to date? Say one machine is always the upload point and the rest get synced from that one? I am having a hard time asking really good questions I think because there are so many things I am trying to think out. Thanks for any ideas, Eric http://www.kwinternet.com/eric (250) 655 - 9513 (PST Time Zone) Inquiry is fatal to certainty. -- Will Durant
Re: Building high load mod_perl/Mason servers
We are going to be moving to mod_perl, but I am worried about how to keep from getting into the same kind of trap with mod_perl as with PHP. You may want to read my article about building a large site with mod_perl: http://perl.apache.org/release/docs/tutorials/apps/scale_etoys/etoys.htm l. So I am thinking whatever I do it should fit within an existing framework, something like Mason. But I am confused about what real advatage Mason provides, and how things like source code control would work if we are running lots of servers. Your best bet to avoid spaghetti code with Mason is to use OO perl modules for all of the real functionality in the application and then use Mason's components as templates for displaying the results. Mason is good at handling the plumbing aspects of web development and provides an environment for doing HTML templates with in-line perl. See this article for more on appropriate use of components and modules: http://masonhq.com/user/autarch/Comps_vs_modules There are many frameworks for mod_perl, so if Mason doesn't look like exactly what you want you can check out some others here: http://perl.apache.org/#appservers Do people use rsync to keep up to date? If you have a high-volume commercial site, you would be better off with a slightly more structured process. You can set up a script to make releases which will tag the CVS tree and build a release in some package format like RPM or .tar.gz. Then you can QA that release and later install the same release, all with development continuing in CVS for the next release. This allows you to easilly go back to a prior release if a disastrous bug is found in production. (Well, there are issues like config files and database changes which complicate things, but that's the basic idea.) - Perrin
RE: Building high load mod_perl/Mason servers
Eric, What we do with our PHP/mod_perl boxes is develop everything locally on development servers and then export it all using rsync. We have six servers in our cluster that we rsync to. Basically, we store everything in a local CVS repository. When changes are made on the local dev boxes and any script is updated, they are checked into the CVS. Then we run a script that syncs from the local CVS to the remote servers (cvs checkout to a temp dir, then rsync from the temp dir to the live servers). It has been working very well so far. Jeremy -Original Message- From: Eric Frazier [mailto:[EMAIL PROTECTED]] Sent: Thursday, June 06, 2002 7:44 PM To: [EMAIL PROTECTED] Subject: Building high load mod_perl/Mason servers Hi, I just got the chance to be in charge of a very busy website, and its future growth. Currently it is running with PHP on two round robin servers with a separate mysql db server. We are going to be moving to mod_perl, but I am worried about how to keep from getting into the same kind of trap with mod_perl as with PHP. The PHP guys don't know OOP, so we have to code halfway, modules exporter but not OOP modules. It has to be something OOP like without getting too complex at first. The PHP trap is just the horrible require once stuff all over the place and global vars etc. I know lots of people blame this kind of coding on perl geeks, but the PHP stuff I have been seeing is pretty bad with it because the constant thought is must fix current problem wait till later to be pretty but later never comes. Also things like using ten instr functions instead of one reg exp. So I am thinking whatever I do it should fit within an existing framework, something like Mason. But I am confused about what real advatage Mason provides, and how things like source code control would work if we are running lots of servers. Do people use rsync to keep up to date? Say one machine is always the upload point and the rest get synced from that one? I am having a hard time asking really good questions I think because there are so many things I am trying to think out. Thanks for any ideas, Eric http://www.kwinternet.com/eric (250) 655 - 9513 (PST Time Zone) Inquiry is fatal to certainty. -- Will Durant
Building high load mod_perl/Mason servers
Hi, I just got the chance to be in charge of a very busy website, and its future growth. Currently it is running with PHP on two round robin servers with a separate mysql db server. We are going to be moving to mod_perl, but I am worried about how to keep from getting into the same kind of trap with mod_perl as with PHP. The PHP guys don't know OOP, so we have to code halfway, modules exporter but not OOP modules. It has to be something OOP like without getting too complex at first. The PHP trap is just the horrible require once stuff all over the place and global vars etc. I know lots of people blame this kind of coding on perl geeks, but the PHP stuff I have been seeing is pretty bad with it because the constant thought is must fix current problem wait till later to be pretty but later never comes. Also things like using ten instr functions instead of one reg exp. So I am thinking whatever I do it should fit within an existing framework, something like Mason. But I am confused about what real advatage Mason provides, and how things like source code control would work if we are running lots of servers. Do people use rsync to keep up to date? Say one machine is always the upload point and the rest get synced from that one? I am having a hard time asking really good questions I think because there are so many things I am trying to think out. Thanks for any ideas, Eric http://www.kwinternet.com/eric (250) 655 - 9513 (PST Time Zone) Inquiry is fatal to certainty. -- Will Durant
