Author: autarch Date: Wed Jul 5 11:15:21 2006 New Revision: 419305 URL: http://svn.apache.org/viewvc?rev=419305&view=rev Log: Update README with pod2text. I wish this were automated cause I'm sure I'll forget on a semi-regular basis.
Modified: perl/Apache-SizeLimit/trunk/README Modified: perl/Apache-SizeLimit/trunk/README URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/README?rev=419305&r1=419304&r2=419305&view=diff ============================================================================== --- perl/Apache-SizeLimit/trunk/README (original) +++ perl/Apache-SizeLimit/trunk/README Wed Jul 5 11:15:21 2006 @@ -1,293 +1,287 @@ NAME - -Apache::SizeLimit - Because size does matter. + Apache::SizeLimit - Because size does matter. SYNOPSIS + <Perl> + Apache::SizeLimit->set_max_process_size(150_000); # Max size in KB + Apache::SizeLimit->set_min_shared_size(10_000); # Min share in KB + Apache::SizeLimit->set_max_unshared_size(120_000); # Max unshared size in KB + </Perl> - <Perl> - $Apache::SizeLimit::MAX_UNSHARED_SIZE = 120000; # 120MB - </Perl> - - PerlCleanupHandler Apache::SizeLimit + PerlCleanupHandler Apache::SizeLimit DESCRIPTION - -This module allows you to kill off Apache httpd processes if they grow -too large. You can make the decision to kill a process based on its -overall size, by setting a minimum limit on shared memory, or a -maximum on unshared memory. - -You can set limits for each of these sizes, and if any limit is not -met, the process will be killed. - -You can also limit the frequency that these sizes are checked so that -this module only checks every N requests. - -This module is highly platform dependent, please read the CAVEATS -section. + This module allows you to kill off Apache httpd processes if they grow + too large. You can make the decision to kill a process based on its + overall size, by setting a minimum limit on shared memory, or a maximum + on unshared memory. + + You can set limits for each of these sizes, and if any limit is + exceeded, the process will be killed. + + You can also limit the frequency that these sizes are checked so that + this module only checks every N requests. + + This module is highly platform dependent, please read the "PER-PLATFORM + BEHAVIOR" section for details. It is possible that this module simply + does not support your platform. API + You can set set the size limits from a Perl module or script loaded by + Apache by calling the appropriate class method on "Apache::SizeLimit": -You can set set the size limits from a Perl module or script loaded by -Apache: - - use Apache::SizeLimit; - - Apache::SizeLimit::setmax(150_000); # Max size in KB - Apache::SizeLimit::setmin(10_000); # Min share in KB - Apache::SizeLimit::setmax_unshared(120_000); # Max unshared size in KB - -Then in your Apache configuration, make Apache::SizeLimit a -C<PerlCleanupHandler>: + * Apache::SizeLimit->set_max_process_size($size) + This sets the maximum size of the process, including both shared and + unshared memory. - PerlCleanupHandler Apache::SizeLimit + * Apache::SizeLimit->set_max_unshared_size($size) + This sets the maximum amount of *unshared* memory the process can + use. -If you want to use C<Apache::SizeLimit> from a registry script, you -must call one of the above functions for every request: + * Apache::SizeLimit->set_min_shared_size($size) + This sets the minimum amount of shared memory the process must have. - use Apache::SizeLimit + The two methods related to shared memory size are effectively a no-op if + the module cannot determine the shared memory size for your platform. + See "PER-PLATFORM BEHAVIOR" for more details. - main(); + Running the handler() + There are several ways to make this module actually run the code to kill + a process. - sub { - Apache::SizeLimit::setmax(150_000); + The simplest is to make "Apache::SizeLimit" a "PerlCleanupHandler" in + your Apache config: - # handle request - }; + PerlCleanupHandler Apache::SizeLimit -Calling any one of C<setmax()>, C<setmin()>, or C<setmax_unshared()> -will install C<Apache::SizeLimit> as a cleanup handler, if it's not -already installed. + This will ensure that "Apache::SizeLimit->handler()" is called run for + all requests. -If you want to combine this module with a cleanup handler of your own, -make sure that C<Apache::SizeLimit> is the last handler run: + If you want to combine this module with a cleanup handler of your own, + make sure that "Apache::SizeLimit" is the last handler run: - PerlCleanupHandler Apache::SizeLimit My::CleanupHandler + PerlCleanupHandler Apache::SizeLimit My::CleanupHandler -Remember, mod_perl will run stacked handlers from right to left, as -they're defined in your configuration. + Remember, mod_perl will run stacked handlers from right to left, as + they're defined in your configuration. -You can explicitly call the C<Apache::SizeLimit::handler()> function -from your own handler: + You can also explicitly call the "Apache::SizeLimit->handler()" function + from your own cleanup handler: - package My::CleanupHandler + package My::CleanupHandler - sub handler { - my $r = shift; + sub handler { + my $r = shift; - # do my thing + # Causes File::Temp to remove any temp dirs created during the + # request + File::Temp::cleanup(); - return Apache::SizeLimit::handler($r); - } + return Apache::SizeLimit->handler($r); + } -Since checking the process size can take a few system calls on some -platforms (e.g. linux), you may want to only check the process size -every N times. To do so, simple set the -C<$Apache::SizeLimit::CHECK_EVERY_N_REQUESTS> global. + * Apache::SizeLimit->add_cleanup_handler($r) + You can call this method inside a request to run + "Apache::SizeLimit"'s "handler()" method for just that request. If + this method is called repeatedly, it ensures that it only every adds + one cleanup handler. - $Apache::SizeLimit::CHECK_EVERY_N_REQUESTS = 2; + Checking Every N Requests + Since checking the process size can take a few system calls on some + platforms (e.g. linux), you may not want to check the process size for + every request. -Now C<Apache::SizeLimit> will only check the process size on every -other request. + * Apache::SizeLimit->set_check_interval($interval) + Calling this causes "Apache::SizeLimit" to only check the process + size every $interval requests. If you want this to affect all + processes, make sure to call this during server startup. -Deprecated API - -Previous versions of this module documented three globals for defining -memory size limits: - - $Apache::SizeLimit::MAX_PROCESS_SIZE - - $Apache::SizeLimit::MIN_SHARE_SIZE - - $Apache::SizeLimit::MAX_UNSHARED_SIZE - -Direct use of these globals is deprecated, but will continue to work -for the foreseeable future. +SHARED MEMORY OPTIONS + In addition to simply checking the total size of a process, this module + can factor in how much of the memory used by the process is actually + being shared by copy-on-write. If you don't understand how memory is + shared in this way, take a look at the mod_perl docs at + http://perl.apache.org/docs/. + + You can take advantage of the shared memory information by setting a + minimum shared size and/or a maximum unshared size. Experience on one + heavily trafficked mod_perl site showed that setting maximum unshared + size and leaving the others unset is the most effective policy. This is + because it only kills off processes that are truly using too much + physical RAM, allowing most processes to live longer and reducing the + process churn rate. + +PER-PLATFORM BEHAVIOR + This module is highly platform dependent, since finding the size of a + process is different for each OS, and some platforms may not be + supported. In particular, the limits on minimum shared memory and + maximum shared memory are currently only supported on Linux and BSD. If + you can contribute support for another OS, patches are very welcome. + + Currently supported OSes: + + linux + For linux we read the process size out of /proc/self/statm. If you are + worried about performance, you can consider using + "Apache::SizeLimit->set_check_interval()" to reduce how often this read + happens. + + As of linux 2.6, /proc/self/statm does not report the amount of memory + shared by the copy-on-write mechanism as shared memory. This means that + decisions made based on shared memory as reported by that interface are + inherently wrong. + + However, as of the 2.6.14 release of the kernel, there is + /proc/self/smaps entry for each process. /proc/self/smaps reports + various sizes for each memory segment of a process and allows us to + count the amount of shared memory correctly. + + If "Apache::SizeLimit" detects a kernel that supports /proc/self/smaps + and the "Linux::Smaps" module is installed it will use that module + instead of /proc/self/statm. + + Reading /proc/self/smaps is expensive compared to /proc/self/statm. It + must look at each page table entry of a process. Further, on + multiprocessor systems the access is synchronized with spinlocks. Again, + you might consider using "Apache::SizeLimit->set_check_interval()". + + Copy-on-write and Shared Memory + The following example shows the effect of copy-on-write: + + <Perl> + require Apache::SizeLimit; + package X; + use strict; + use Apache::Constants qw(OK); + + my $x = "a" x (1024*1024); + + sub handler { + my $r = shift; + my ($size, $shared) = $Apache::SizeLimit->_check_size(); + $x =~ tr/a/b/; + my ($size2, $shared2) = $Apache::SizeLimit->_check_size(); + $r->content_type('text/plain'); + $r->print("1: size=$size shared=$shared\n"); + $r->print("2: size=$size2 shared=$shared2\n"); + return OK; + } + </Perl> + + <Location /X> + SetHandler modperl + PerlResponseHandler X + </Location> + + The parent Apache process allocates memory for the string in $x. The + "tr"-command then overwrites all "a" with "b" if the handler is called + with an argument. This write is done in place, thus, the process size + doesn't change. Only $x is not shared anymore by means of copy-on-write + between the parent and the child. + + If /proc/self/smaps is available curl shows: + + [EMAIL PROTECTED]:~/work/mp2> curl http://localhost:8181/X + 1: size=13452 shared=7456 + 2: size=13452 shared=6432 + + Shared memory has lost 1024 kB. The process' overall size remains + unchanged. + + Without /proc/self/smaps it says: + + [EMAIL PROTECTED]:~/work/mp2> curl http://localhost:8181/X + 1: size=13052 shared=3628 + 2: size=13052 shared=3636 + + One can see the kernel lies about the shared memory. It simply doesn't + count copy-on-write pages as shared. + + solaris 2.6 and above + For solaris we simply retrieve the size of /proc/self/as, which contains + the address-space image of the process, and convert to KB. Shared memory + calculations are not supported. + + NOTE: This is only known to work for solaris 2.6 and above. Evidently + the /proc filesystem has changed between 2.5.1 and 2.6. Can anyone + confirm or deny? + + BSD (and OSX) + Uses "BSD::Resource::getrusage()" to determine process size. This is + pretty efficient (a lot more efficient than reading it from the /proc fs + anyway). + + AIX? + Uses "BSD::Resource::getrusage()" to determine process size. Not sure if + the shared memory calculations will work or not. AIX users? + + Win32 + Uses "Win32::API" to access process memory information. "Win32::API" can + be installed under ActiveState perl using the supplied ppm utility. + + Everything Else + If your platform is not supported, then please send a patch to check the + process size. The more portable/efficient/correct the solution the + better, of course. ABOUT THIS MODULE + This module was written in response to questions on the mod_perl mailing + list on how to tell the httpd process to exit if it gets too big. -This module was written in response to questions on the mod_perl -mailing list on how to tell the httpd process to exit if it gets too -big. - -Actually, there are two big reasons your httpd children will grow. -First, your code could have a bug that causes the process to increase -in size very quickly. Second, you could just be doing operations that -require a lot of memory for each request. Since Perl does not give -memory back to the system after using it, the process size can grow -quite large. - -This module will not really help you with the first problem. For that -you should probably look into C<Apache::Resource> or some other means -of setting a limit on the data size of your program. BSD-ish systems -have C<setrlimit()>, which will kill your memory gobbling processes. -However, it is a little violent, terminating your process in -mid-request. - -This module attempts to solve the second situation, where your process -slowly grows over time. It checks memory usage after every request, -and if it exceeds a threshold, exits gracefully. - -By using this module, you should be able to discontinue using the -Apache configuration directive B<MaxRequestsPerChild>, although for -some folks, using both in combination does the job. - -SHARED MEMORY OPTIONS + Actually, there are two big reasons your httpd children will grow. + First, your code could have a bug that causes the process to increase in + size very quickly. Second, you could just be doing operations that + require a lot of memory for each request. Since Perl does not give + memory back to the system after using it, the process size can grow + quite large. + + This module will not really help you with the first problem. For that + you should probably look into "Apache::Resource" or some other means of + setting a limit on the data size of your program. BSD-ish systems have + "setrlimit()", which will kill your memory gobbling processes. However, + it is a little violent, terminating your process in mid-request. + + This module attempts to solve the second situation, where your process + slowly grows over time. It checks memory usage after every request, and + if it exceeds a threshold, exits gracefully. + + By using this module, you should be able to discontinue using the Apache + configuration directive MaxRequestsPerChild, although for some folks, + using both in combination does the job. + +DEPRECATED APIS + Previous versions of this module documented three globals for defining + memory size limits: + + * $Apache::SizeLimit::MAX_PROCESS_SIZE + * $Apache::SizeLimit::MIN_SHARE_SIZE + * $Apache::SizeLimit::MAX_UNSHARED_SIZE + * $Apache::SizeLimit::CHECK_EVERY_N_REQUESTS + * $Apache::SizeLimit::USE_SMAPS + + Direct use of these globals is deprecated, but will continue to work for + the foreseeable future. + + It also documented three functions for use from registry scripts: + + * Apache::SizeLimit::setmax() + * Apache::SizeLimit::setmin() + * Apache::SizeLimit::setmax_unshared() -In addition to simply checking the total size of a process, this -module can factor in how much of the memory used by the process is -actually being shared by copy-on-write. If you don't understand how -memory is shared in this way, take a look at the mod_perl Guide at -http://perl.apache.org/guide/. - -You can take advantage of the shared memory information by setting a -minimum shared size and/or a maximum unshared size. Experience on one -heavily trafficked mod_perl site showed that setting maximum unshared -size and leaving the others unset is the most effective policy. This -is because it only kills off processes that are truly using too much -physical RAM, allowing most processes to live longer and reducing the -process churn rate. - -CAVEATS - -This module is highly platform dependent, since finding the size of a -process is different for each OS, and some platforms may not be -supported. In particular, the limits on minimum shared memory and -maximum shared memory are currently only supported on Linux and BSD. -If you can contribute support for another OS, patches are very -welcome. - -Currently supported OSes: - -linux - -For linux we read the process size out of F</proc/self/statm>. This -is a little slow, but usually not too bad. If you are worried about -performance, try only setting up the the exit handler inside CGIs -(with the C<setmax()> function), and see if the CHECK_EVERY_N_REQUESTS -option is of benefit. - -Since linux 2.6 F</proc/self/statm> does not report the amount of -memory shared by the copy-on-write mechanism as shared memory. Hence -decisions made on the basis of C<MAX_UNSHARED_SIZE> or -C<MIN_SHARE_SIZE> are inherently wrong. - -To correct this situation, as of the 2.6.14 release of the kernel, -there is F</proc/self/smaps> entry for each -process. F</proc/self/smaps> reports various sizes for each memory -segment of a process and allows us to count the amount of shared -memory correctly. - -If C<Apache::SizeLimit> detects a kernel that supports -F</proc/self/smaps> and if the C<Linux::Smaps> module is installed it -will use them instead of F</proc/self/statm>. You can prevent -C<Apache::SizeLimit> from using F</proc/self/smaps> and turn on the -old behaviour by setting C<$Apache::SizeLimit::USE_SMAPS> to 0. - -C<Apache::SizeLimit> itself will C<$Apache::SizeLimit::USE_SMAPS> to 0 -if it cannot load C<Linux::Smaps> or if your kernel does not support -F</proc/self/smaps>. Thus, you can check it to determine what is -actually used. - -NOTE: Reading F</proc/self/smaps> is expensive compared to -F</proc/self/statm>. It must look at each page table entry of a process. -Further, on multiprocessor systems the access is synchronized with -spinlocks. Hence, you are encouraged to set the C<CHECK_EVERY_N_REQUESTS> -option. - -The following example shows the effect of copy-on-write: - - <Perl> - require Apache::SizeLimit; - package X; - use strict; - use Apache::Constants qw(OK); - - my $x= "a" x (1024*1024); - - sub handler { - my $r = shift; - my ($size, $shared) = $Apache::SizeLimit::check_size(); - $x =~ tr/a/b/; - my ($size2, $shared2) = $Apache::SizeLimit::check_size(); - $r->content_type('text/plain'); - $r->print("1: size=$size shared=$shared\n"); - $r->print("2: size=$size2 shared=$shared2\n"); - return OK; - } - </Perl> - - <Location /X> - SetHandler modperl - PerlResponseHandler X - </Location> - -The parent apache allocates a megabyte for the string in C<$x>. The -C<tr>-command then overwrites all "a" with "b" if the handler is -called with an argument. This write is done in place, thus, the -process size doesn't change. Only C<$x> is not shared anymore by -means of copy-on-write between the parent and the child. - -If F</proc/self/smaps> is available curl shows: - - [EMAIL PROTECTED]:~/work/mp2> curl http://localhost:8181/X - 1: size=13452 shared=7456 - 2: size=13452 shared=6432 - -Shared memory has lost 1024 kB. The process' overall size remains unchanged. - -Without F</proc/self/smaps> it says: - - [EMAIL PROTECTED]:~/work/mp2> curl http://localhost:8181/X - 1: size=13052 shared=3628 - 2: size=13052 shared=3636 - -One can see the kernel lies about the shared memory. It simply doesn't -count copy-on-write pages as shared. - -solaris 2.6 and above - -For solaris we simply retrieve the size of F</proc/self/as>, which -contains the address-space image of the process, and convert to KB. -Shared memory calculations are not supported. - -NOTE: This is only known to work for solaris 2.6 and above. Evidently -the F</proc> filesystem has changed between 2.5.1 and 2.6. Can anyone -confirm or deny? - -*bsd* - -Uses C<BSD::Resource::getrusage()> to determine process size. This is -pretty efficient (a lot more efficient than reading it from the -F</proc> fs anyway). - -AIX? - -Uses C<BSD::Resource::getrusage()> to determine process size. Not -sure if the shared memory calculations will work or not. AIX users? - -Win32 - -Uses C<Win32::API> to access process memory information. -C<Win32::API> can be installed under ActiveState perl using the -supplied ppm utility. - -If your platform is not supported, then please send a patch to check -the process size. The more portable/efficient/correct the solution the -better, of course. + Besides setting the appropriate limit, these functions *also* add a + cleanup handler to the current request. AUTHOR + Doug Bagley <[EMAIL PROTECTED]>, channeling Procrustes. -Doug Bagley <[EMAIL PROTECTED]>, channeling Procrustes. - -Brian Moseley <[EMAIL PROTECTED]>: Solaris 2.6 support + Brian Moseley <[EMAIL PROTECTED]>: Solaris 2.6 support -Doug Steinwand and Perrin Harkins <[EMAIL PROTECTED]>: added support - for shared memory and additional diagnostic info + Doug Steinwand and Perrin Harkins <[EMAIL PROTECTED]>: added support for + shared memory and additional diagnostic info -Matt Phillips <[EMAIL PROTECTED]> and Mohamed Hendawi -<[EMAIL PROTECTED]>: Win32 support + Matt Phillips <[EMAIL PROTECTED]> and Mohamed Hendawi + <[EMAIL PROTECTED]>: Win32 support -Dave Rolsky <[EMAIL PROTECTED]>, maintenance and fixes outside of -mod_perl tree (0.06). + Dave Rolsky <[EMAIL PROTECTED]>, maintenance and fixes outside of + mod_perl tree (0.9+).