Re: reducing size of apache instances
John Almberg wrote: I'm starting to wonder about the Swap info from top... it never changes. It has said the same thing all day, since I've been watching it. Does that make sense? Swap: 2008M Total, 150M Used, 1858M Free, 7% Inuse That looks about normal if your RAM suffices. In that case only memory that hasn't been accessed for more than 24 hours will be moved to the swap space. So after an uptime of 24 hours, if no more apps get started there's no more reason for it to change. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
Here's a few you can disable: mod_status, mod_info (both give extra unnecessary info about server) and mod_include (allows include statements in html files). These aren't recommended anyway unless you really need them as they create some level of security concern. mod_userdir is only needed if you are allowing users to have webpages from home directories mod_ssl, only if you're going to use ssl mod_rewrite only if you will be rewriting webpage names to something else mod_auth_basic and mod_auth_digest only if you're going to add http authentication to any of your webpages John Almberg wrote: My Apache 2.2 instances are running about 18 Meg each. I've been thinking about doing something to trim these down, and I think tomorrow is the day to take action. They are getting out of hand. I've done a bit of research on this. I think the way to get started is to eliminate unused modules. Problem is, I know which ones I need, since I purposefully added them. I *don't* know which ones I don't need, if you see what I mean, since I inherited them from the default configuration. I assume that some are critical to the basic operation of Apache. I am hoping I can google a list of these tomorrow. Obviously these I'll have to live with. But what about the set that is left after I remove the ones the system needs, and the ones I need? How do I know which ones I can safely turn off? All I can think of is a trial and error process (i.e., turn them off one by one and see if anything breaks.) Is there a better way? -- John ___ [1]freebsd-questi...@freebsd.org mailing list [2]http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [3]freebsd-questions-unsubscr...@freebsd.org References 1. mailto:freebsd-questions@freebsd.org 2. http://lists.freebsd.org/mailman/listinfo/freebsd-questions 3. mailto:freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
John Almberg wrote: My Apache 2.2 instances are running about 18 Meg each. I've been thinking about doing something to trim these down, and I think tomorrow is the day to take action. They are getting out of hand. I've done a bit of research on this. I think the way to get started is to eliminate unused modules. Problem is, I know which ones I need, since There is another thing you can try. Judging from the process size you've given it looks like you are not using PHP or a similar Apache module. Also, you didn't specify anything so I assume you are using the default configuration, which operates in prefork mode - MPM_PREFORK, which means a separate process is forked for every request. If all of this is true, you can trivially switch to the worker-threaded MPM in which every Apache process (which will be of comparable size to the one you currently have) will handle a large number of request. In effect, instead of e.g. 50 Apache processes active for 50 connections, you will have 2-3 Apache processes. Enable WITH_MPM=worker in /etc/make.conf to enable this. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
Ivan Voras wrote: There is another thing you can try. Judging from the process size you've given it looks like you are not using PHP or a similar Apache module. Also, you didn't specify anything so I assume you are using the default configuration, which operates in prefork mode - MPM_PREFORK, which means a separate process is forked for every request. If all of this is true, you can trivially switch to the worker-threaded MPM in which every Apache process (which will be of comparable size to the one you currently have) will handle a large number of request. In effect, instead of e.g. 50 Apache processes active for 50 connections, you will have 2-3 Apache processes. Enable WITH_MPM=worker in /etc/make.conf to enable this. I am using PHP, in fact. I've listed all the loaded modules below, and marked the ones I added with an '*'. I need the proxy modules because I use Apache as a front end for Mongrel. BTW, this is Apache 2.2 on FreeBSD 7.1 This WITH_MPM=worker sounds interesting. I'll have to read up on it. I guess there is some downside to enabling it, like slower performance? -- John Loaded Modules: core_module (static) mpm_prefork_module (static) http_module (static) so_module (static) authn_file_module (shared) authn_dbm_module (shared) authn_anon_module (shared) authn_default_module (shared) authn_alias_module (shared) authz_host_module (shared) authz_groupfile_module (shared) authz_user_module (shared) authz_dbm_module (shared) authz_owner_module (shared) authz_default_module (shared) auth_basic_module (shared) auth_digest_module (shared) file_cache_module (shared) cache_module (shared) disk_cache_module (shared) dumpio_module (shared) include_module (shared) filter_module (shared) charset_lite_module (shared) deflate_module (shared) log_config_module (shared) logio_module (shared) env_module (shared) mime_magic_module (shared) cern_meta_module (shared) expires_module (shared) headers_module (shared) usertrack_module (shared) unique_id_module (shared) setenvif_module (shared) version_module (shared) *proxy_module (shared) *proxy_http_module (shared) *proxy_balancer_module (shared) *ssl_module (shared) mime_module (shared) *dav_module (shared) status_module (shared) autoindex_module (shared) asis_module (shared) info_module (shared) cgi_module (shared) dav_fs_module (shared) vhost_alias_module (shared) negotiation_module (shared) dir_module (shared) imagemap_module (shared) actions_module (shared) speling_module (shared) userdir_module (shared) alias_module (shared) *rewrite_module (shared) *php5_module (shared) Syntax OK ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
John Almberg wrote: I am using PHP, in fact. I've listed all the loaded modules below, and marked the ones I added with an '*'. I need the proxy modules because I use Apache as a front end for Mongrel. This WITH_MPM=worker sounds interesting. I'll have to read up on it. I guess there is some downside to enabling it, like slower performance? PHP is incredibly buggy and will in all probability break Apache if you try running it in threaded mode. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
PHP is incredibly buggy and will in all probability break Apache if you try running it in threaded mode. That doesn't sound so good. As a sanity check... I've been studying these processes all morning. When I use 'top', the column RES shows the amount of RAM used for the process, correct? This is the value I'd like to get down. Okay, well after a morning studying and observing, and thanks to suggestions from you all, I think I understand enough to start turning modules off. Crossing fingers... -- John ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
On Fri, Sep 11, 2009 at 1:48 PM, John Almberg jalmb...@identry.com wrote: As a sanity check... I've been studying these processes all morning. When I use 'top', the column RES shows the amount of RAM used for the process, correct? This is the value I'd like to get down. How many Apache processes are involved, total? Because I'm really not sure how much success you're going to have with this. You're at 22mb already (by comparison mine are 44mb *without* mod_php). How much improvement are you looking for? A couple of megs? Unless there are tens of thousands of processes, buying a couple of gigs of RAM is probably the most time and cost effective solution. Also keep in mind that a lot of the RES is the configuration, which isn't going to change at all when you disable modules. (Unless you change it.) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
On Fri, Sep 11, 2009 at 1:05 PM, Linda Messerschmidt linda.messerschm...@gmail.com wrote: On Fri, Sep 11, 2009 at 1:48 PM, John Almberg jalmb...@identry.com wrote: As a sanity check... I've been studying these processes all morning. When I use 'top', the column RES shows the amount of RAM used for the process, correct? This is the value I'd like to get down. How many Apache processes are involved, total? Because I'm really not sure how much success you're going to have with this. You're at 22mb already (by comparison mine are 44mb *without* mod_php). How much improvement are you looking for? A couple of megs? Unless there are tens of thousands of processes, buying a couple of gigs of RAM is probably the most time and cost effective solution. Also keep in mind that a lot of the RES is the configuration, which isn't going to change at all when you disable modules. (Unless you change it.) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org Not to mention the RES column is a horribly inaccurate method of calculating mem usage by application. They don't run 18 MB each that is shared mem. -- Adam Vande More ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
Ivan Voras wrote: John Almberg wrote: I am using PHP, in fact. I've listed all the loaded modules below, and marked the ones I added with an '*'. I need the proxy modules because I use Apache as a front end for Mongrel. This WITH_MPM=worker sounds interesting. I'll have to read up on it. I guess there is some downside to enabling it, like slower performance? PHP is incredibly buggy and will in all probability break Apache if you try running it in threaded mode. Try mod_fcgid and run PHP as FastCGI if you want a threaded Apache. I've been running the event mpm on a test box at work this way. So far I've never had any trouble, but it is also not getting hammered either. Conventional wisdom is the event mpm is still considered experimental and therefore untrustworthy in a production environment. -Mike ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
Linda Messerschmidt wrote: On Fri, Sep 11, 2009 at 1:48 PM, John Almbergjalmb...@identry.com wrote: As a sanity check... I've been studying these processes all morning. When I use 'top', the column RES shows the amount of RAM used for the process, correct? This is the value I'd like to get down. How many Apache processes are involved, total? Because I'm really not sure how much success you're going to have with this. You're at 22mb already (by comparison mine are 44mb *without* mod_php). How much improvement are you looking for? A couple of megs? Yup... that's about what I got for my troubles. After turning off all the unneeded modules, they are now running about 17mb. Not a huge improvement... I definitely need more ram and I have it on order. While I'm waiting for it, I figured I'd see what processes I could slim down. My basic problem is at peak usage times (usually in the afternoon), the server starts using swap space, and then response times really bog down. This is on a 'spare' server that is temporarily in service while one of our 'big' servers is out for repair. This 'spare' server only has 1G ram and was never really meant for web server service. It's trying it's best. I'm just trying to lighten the load for it. -- John ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
Hi-- On Sep 11, 2009, at 12:42 PM, John Almberg wrote: My basic problem is at peak usage times (usually in the afternoon), the server starts using swap space, and then response times really bog down. Limit the MaxChildren to the number of Apache httpd's which your machine can actually handle. You may (will!) still get some backlog, but keeping the system from swapping will help responsiveness Regards, -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
On Fri, Sep 11, 2009 at 2:42 PM, John Almberg jalmb...@identry.com wrote: Linda Messerschmidt wrote: On Fri, Sep 11, 2009 at 1:48 PM, John Almbergjalmb...@identry.com wrote: As a sanity check... I've been studying these processes all morning. When I use 'top', the column RES shows the amount of RAM used for the process, correct? This is the value I'd like to get down. How many Apache processes are involved, total? Because I'm really not sure how much success you're going to have with this. You're at 22mb already (by comparison mine are 44mb *without* mod_php). How much improvement are you looking for? A couple of megs? Yup... that's about what I got for my troubles. After turning off all the unneeded modules, they are now running about 17mb. Not a huge improvement... I definitely need more ram and I have it on order. While I'm waiting for it, I figured I'd see what processes I could slim down. My basic problem is at peak usage times (usually in the afternoon), the server starts using swap space, and then response times really bog down. This is on a 'spare' server that is temporarily in service while one of our 'big' servers is out for repair. This 'spare' server only has 1G ram and was never really meant for web server service. It's trying it's best. I'm just trying to lighten the load for it. -- John You've misunderstood what you've done. You have not saved a couple of MB, you've saved one. Of the 18 MB, nearly all of it is shared memory which is only loaded once. This is a bit hypothetical but take the following 1160 vandemorea 1 460 18M 83108K select 1 51:18 2.39% httpd 1482 vandemorea 11 440 18M 220M ucond 1 104:47 2.20% httpd 37776 vandemorea 20 440 18M 1179M select 1 95:43 1.76% httpd 1301 vandemorea1 450 18M 28856K select 1 167:00 1.07% httpd 1311 vandemorea2 470 18M 128M select 1 53:39 0.68% httpd 1407 vandemorea2 440 18M 49284K select 1 3:41 0.20% httpd 38613 vandemorea1 440 18M 31220K select 0 8:16 0.00% httpd 1320 vandemorea2 440 18M 53788K ucond 1 5:20 0.00% httpd 5455 vandemorea1 440 18M 1292K select 0 4:26 0.00% httpd 64974 vandemorea 21 520 18M 241M ucond 0 4:20 0.00% httpd from your posts, I understand you read that as httpd using a total of 180 MB of RAM. This is incorrect. The RES column also included shared mem which is only loaded once but counted on each line. So if the size of the shared mem in use by httpd is 17MB, in total it would be consuming 27MB, not the 180. So chasing memory savings here is probably not pursuing low hanging fruit. 1GB web server is more than enough for basic www server, even more. -- Adam Vande More ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
You've misunderstood what you've done. You have not saved a couple of MB, you've saved one. Of the 18 MB, nearly all of it is shared memory which is only loaded once. Ah... Okay. That actually makes sense. Thanks for the clarification. 1GB web server is more than enough for basic www server, even more. I would have thought, but some times it really gets slow and I'm trying to figure out why. When bogged down, the load averages are low. The main thing that looks out of whack is swap space, which seems to never go below 7%, but sometimes gets up into the 20%-30% range. When it gets that high, the server slows to a crawl. last pid: 12732; load averages: 0.44, 0.31, 0.27 up 34+03:57:58 16:16:27 187 processes: 2 running, 185 sleeping CPU: 4.5% user, 0.0% nice, 1.1% system, 0.0% interrupt, 94.4% idle Mem: 425M Active, 106M Inact, 268M Wired, 3160K Cache, 110M Buf, 176M Free Swap: 2008M Total, 150M Used, 1858M Free, 7% Inuse ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
Have given Nginx web server a try? It is small and may work better with limited RAM. http://www.nginx.net/ http://urloid.com/nginx1 Diego 2009/9/11 John Almberg jalmb...@identry.com: You've misunderstood what you've done. You have not saved a couple of MB, you've saved one. Of the 18 MB, nearly all of it is shared memory which is only loaded once. Ah... Okay. That actually makes sense. Thanks for the clarification. 1GB web server is more than enough for basic www server, even more. I would have thought, but some times it really gets slow and I'm trying to figure out why. When bogged down, the load averages are low. The main thing that looks out of whack is swap space, which seems to never go below 7%, but sometimes gets up into the 20%-30% range. When it gets that high, the server slows to a crawl. last pid: 12732; load averages: 0.44, 0.31, 0.27 up 34+03:57:58 16:16:27 187 processes: 2 running, 185 sleeping CPU: 4.5% user, 0.0% nice, 1.1% system, 0.0% interrupt, 94.4% idle Mem: 425M Active, 106M Inact, 268M Wired, 3160K Cache, 110M Buf, 176M Free Swap: 2008M Total, 150M Used, 1858M Free, 7% Inuse ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
On Fri, Sep 11, 2009 at 3:20 PM, John Almberg jalmb...@identry.com wrote: You've misunderstood what you've done. You have not saved a couple of MB, you've saved one. Of the 18 MB, nearly all of it is shared memory which is only loaded once. Ah... Okay. That actually makes sense. Thanks for the clarification. 1GB web server is more than enough for basic www server, even more. I would have thought, but some times it really gets slow and I'm trying to figure out why. When bogged down, the load averages are low. The main thing that looks out of whack is swap space, which seems to never go below 7%, but sometimes gets up into the 20%-30% range. When it gets that high, the server slows to a crawl. last pid: 12732; load averages: 0.44, 0.31, 0.27 up 34+03:57:58 16:16:27 187 processes: 2 running, 185 sleeping CPU: 4.5% user, 0.0% nice, 1.1% system, 0.0% interrupt, 94.4% idle Mem: 425M Active, 106M Inact, 268M Wired, 3160K Cache, 110M Buf, 176M Free Swap: 2008M Total, 150M Used, 1858M Free, 7% Inuse You're on the right track pursuing whatever is consuming swap. Are there any other services running on the system which may be consuming mem? eg postgresql? You can start eliminating unnecessary services, and limiting memory by app. php.ini contains a knob for this. Plus as mentioned before limiting number of connections may help depending on what other things system is running. -- Adam Vande More ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
On Fri, Sep 11, 2009 at 4:20 PM, John Almberg jalmb...@identry.com wrote: I would have thought, but some times it really gets slow and I'm trying to figure out why. When bogged down, the load averages are low. The main thing that looks out of whack is swap space, which seems to never go below 7%, but sometimes gets up into the 20%-30% range. When it gets that high, the server slows to a crawl. In this case you don't want to look at processes with big RES, you want to find processes with a big difference between RES and SIZE and/or the ones with flat-out largest SIZE. Try sorting top by SIZE and see what bubbles up. (Ignore rpc.statd if it's running.) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
In this case you don't want to look at processes with big RES, you want to find processes with a big difference between RES and SIZE and/or the ones with flat-out largest SIZE. Try sorting top by SIZE and see what bubbles up. (Ignore rpc.statd if it's running.) Huh... okay. That's interesting. Well the biggest SIZE process is mysql, followed by three mongrel instances (for a ruby on rails app), and then a bunch of httpd processes. Mysql is optimized for a small server, there isn't much I can do about the size of the Rails app, so the apache instances seemed like the logical place to start. I'm starting to wonder about the Swap info from top... it never changes. It has said the same thing all day, since I've been watching it. Does that make sense? Swap: 2008M Total, 150M Used, 1858M Free, 7% Inuse -- John ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
John Almberg writes: I'm starting to wonder about the Swap info from top... it never changes. It has said the same thing all day, since I've been watching it. Does that make sense? The current machine has 8G, so ... porbably not a good test case. :-) It's predecessor succumbed when it had 2G; depending on what was running, sometimes the swap would stick for hours or even days, Robert Huff ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
In the last episode (Sep 11), John Almberg said: In this case you don't want to look at processes with big RES, you want to find processes with a big difference between RES and SIZE and/or the ones with flat-out largest SIZE. Try sorting top by SIZE and see what bubbles up. (Ignore rpc.statd if it's running.) Huh... okay. That's interesting. Well the biggest SIZE process is mysql, followed by three mongrel instances (for a ruby on rails app), and then a bunch of httpd processes. Mysql is optimized for a small server, there isn't much I can do about the size of the Rails app, so the apache instances seemed like the logical place to start. I'm starting to wonder about the Swap info from top... it never changes. It has said the same thing all day, since I've been watching it. Does that make sense? Swap: 2008M Total, 150M Used, 1858M Free, 7% Inuse If you previously ran some memory-intensive program, the system would have pushed some unused data out to disk ( login processes for unused VTYs, other daemons that are rarely used, etc ), but it won't free up that swap space until those processes exit. As long as you don't see ###K Out, ###K In on that swap line, you're not actively using the swap space and don't have to worry. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
On Sep 10, 2009, at 7:58 PM, John Almberg wrote: My Apache 2.2 instances are running about 18 Meg each. I've been thinking about doing something to trim these down, and I think tomorrow is the day to take action. They are getting out of hand. [ ... ] But what about the set that is left after I remove the ones the system needs, and the ones I need? How do I know which ones I can safely turn off? All I can think of is a trial and error process (i.e., turn them off one by one and see if anything breaks.) Is there a better way? Yes. Figure out which modules you actually need, and only enable those. What modules you are using should be reasonably clear from the access and error logs-- you should be able to see which URLs you are serving, and hence which modules were involved. Regards, -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reducing size of apache instances
On Thu, Sep 10, 2009 at 10:58 PM, John Almberg jalmb...@identry.com wrote: I assume that some are critical to the basic operation of Apache. I am hoping I can google a list of these tomorrow. Obviously these I'll have to live with. This is a pretty short list, and Apache won't start without them. All I can think of is a trial and error process (i.e., turn them off one by one and see if anything breaks.) Is there a better way? Other than those core modules you mentioned above, one of the most distinguishing characteristics of modules is that they define config directives that you then use. I would recommend that you walk through your configs and determine which module each and every directive comes from. To a reasonable degree of accuracy, that will give you the list of modules that are really in use. Naturally you'll find some exception(s), but this will get you very close without a lot of trial-and-error downtime. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
