ID: 43105
User updated by: ian at onlineloop dot com
Reported By: ian at onlineloop dot com
Status: Assigned
Bug Type: Unknown/Other Function
Operating System: Solaris 10
PHP Version: 5.2.5RC2-dev
Assigned To: ab5602
New Comment:
Hi,
I don't want to post the attahced file to the bug report as this may
expose more information that I like about our web server. It is a
pfiles output from the apache processes. I will send the file to you
directly in an email.
The exact situation is difficult to reproduce, and happened most
frequently when the server was under load. We are typically handling
about 100-120 requests per second, and under PHP 5.1.6 (we are back
on it now), there was usually between 500 and 1000 open files at any
one time between the apache processes.
The symptom of the problem - system error messages, masses of open
files, not being able to execute ssmtp and not being able to open
files, also generally exists for just a couple of seconds, then goes
away again. The total memory usage of apache climbs constantly,
usually hitting 6-8Gb within 24 hours of an apache restart, and
continuing to climb if apache is not restarted. After about 36
hours, the ram usage is over 10Gb, at which point I restart the
server as we need it to be available. Since running 5.1.6 again (one
week), the memory usage is constantly around 1.5-2Gb and there are no
problems from 5.2.5rc-x. I fear reproducing this will be difficult
for you.
ssmtp is generally able to send mails, just occasionally it is hit by
this problem. Apache also has problems opening other files under
this condition, apparently mainly for writing. I have tried on the
command line to send mails with ssmtp when the error messages start
coming out, however that worked and the test mails came in. "cat"ing
files into another file, and things like this also worked.
The problem of not being able to open files also occurs when not
trying to send mails, as such my impression the problem is more
general than just being connected with mail function. I did put
sendmail in briefly today to see what happened, the problem still
occured. A restart in the middle of the day for the apache process
is not an option here as the service needs to be available. I can
only make changes to the system between 2 and 4am.
I will do what I can to help pinpoint this problem, please however
understand the restrictions I have as I do this on a productive
system.
Previous Comments:
------------------------------------------------------------------------
[2007-11-11 19:18:31] [EMAIL PROTECTED]
1) What file(s) are being held open? (lsof/ptree may help).
2) Is 'ssmtp -t' successfully opening, delivering the email and closing
it's own network connections under this condition? Every open network
connection requires 2 file descriptors.
3) Does this happen without sending mail()?
4) Does this happen with other delivery programs such as 'sendmail -t'?
Thanks,
-Rob
------------------------------------------------------------------------
[2007-11-11 19:03:23] [EMAIL PROTECTED]
So, how can we reproduce this?
------------------------------------------------------------------------
[2007-11-06 11:19:24] ian at onlineloop dot com
Changing this whole report as I have now more information.
After further examining the problem, it is clear that PHP is sometimes
failing to close the files it opens, and is generally failing to cleanly
exit, if it exits at all. It is running with Apache 2.0.61. As time
passes, PHP has more and more files open, the apache child processes do
not exit, the status of the child processes stays on "W", writing, for
long periods of time although the network status on the server itsel
clearly shows that all connections to the remote address have closed.
The memory usage also climbs constantly until the apache child process
dies, if the child process dies at all.
In the system, this is noticable by the number of open files, in the
last 24 hours, this has changed from 4.2Gb and 30,711 open files to now
6.6Gb and 48,462 open files. This increase in memory consumption and
the number of open files is continuous. Additionally, there are more
and more apache child processes that simply do not exit. It is this
behaviour that is clearly responsible for the problems of opening other
files, and for the failure to be able to execute some system calls, such
as sending mail and writing log files.
The cause for this behaviour is unclear, however it is definately PHP
5.2.x related as under PHP 5.1.6, everything is ok.
------------------------------------------------------------------------
[2007-10-25 15:03:26] ian at onlineloop dot com
Description:
------------
Working with PHP 5.2.5-RC2, build from php5.2-200710241430, problem
also occurred in php5.2-200710171030.
PHP for what ever reason is occasionally unable to execute the mail
program that we have installed, ssmtp. I am able to run the script below
15-20 times, and 1 or 2 times the mail program fails, every other time
it is successful. This problem seesm to occur only when there is a
higher system load, apache needs to be serving 60-80 requests before
this problem shows itself. Suspicion was in the maximum number of open
files, however that can be ruled out as our system (Sun V440, 16Gb RAM,
4 x SPARC 1.6Ghz) allows up to 10240 open files at any one time.
Additionally, there are sometimes problems creating files. Although the
errors would point to a system problem, this is definately not the case
as other processes on the system do not have problems (proftpd, for
example), and everything was OK with PHP 5.1.6.
A log file extract and system messages are:
# Oct 25 16:28:01 www-walda mod_evasive[21333]: [ID 869489
daemon.alert] Couldn't open logfile
/var/apache2/logs/mod_evasive-log/dos-86.56.222.150: Bad file number
[25-Oct-2007 16:39:48] [UseBB Error] [Thu Oct 25 14:39:48 2007]
[E_WARNING - mb_send_mail() [<a
href='function.mb-send-mail'>function.mb-send-mail</a>]: Could not
execute mail delivery program '/usr/bin/ssmtp -t']
[./sources/functions.php:1164]
[25-Oct-2007 16:30:03] PHP Warning: mail() [<a
href='function.mail'>function.mail</a>]: Could not execute mail delivery
program '/usr/bin/ssmtp -t' in /ftp/usr/ian/mailtest.php on line 12
These problems have appeared only since we changed from PHP 5.1.6 to
PHP 5.2.x (note that we have had many problems with PHP 5.2.x). We are
currently using the PHP CVS version mentioned above, as up until and
including PHP 5.2.4, bug numbers 41822, 41899 were blocking us.
Reproduce code:
---------------
<?php
// Mail addresses have been removed because of spam concerns!
$to = '<enter your mail address here';
$subject = 'the testmail';
$message = 'Testmail' ;
$headers = 'From: <senders mail address>' . "\r\n" .
'Reply-To: <senders mail address>' . "\r\n" .
'X-Mailer: PHP/' . phpversion();
$date= date("d M y, G:i:s");
if ( mail($to, $subject, $message, $headers) == TRUE)
echo "OK: $date, mail($to, $subject, $message, $headers)";
else
echo "False";
?>
Expected result:
----------------
The code should run correctly every time, delivering a mail.
Actual result:
--------------
The mail is sent most times, but occasionally I get an error message
back saying that the sending failed:
Browser:
Warning: mail() [function.mail]: Could not execute mail delivery
program '/usr/bin/ssmtp -t' in /ftp/usr/ian/mailtest.php on line 12
False
Logs:
[25-Oct-2007 16:56:58] PHP Warning: mail() [<a
href='function.mail'>function.mail</a>]: Could not execute mail delivery
program '/usr/bin/ssmtp -t' in /ftp/usr/ian/mailtest.php on line 12
------------------------------------------------------------------------
--
Edit this bug report at http://bugs.php.net/?id=43105&edit=1
