[Fwd: Quotes in uploaded filenames are not parsed correctly.]
Lets take this to apreq-dev land.
--
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollu...@p6m7g8.com) c: 703.336.9354
Consultant - P6M7G8 Inc.http://p6m7g8.net
Director IT - RideCharge, Inc. http://ridecharge.com
Contractor - PositiveEnergyUSA http://positiveenergyusa.com
ASF Member - Apache Software Foundation http://apache.org
FreeBSD Committer - FreeBSD Foundation http://freebsd.org
Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.
---BeginMessage---
When you handle a multipart/form-data post with libapreq quotes in
filenames are mishandled. For example, a post that includes:
Content-Disposition: form-data; name=foo; filename=breakhere.jpg
Will result in a filename of just 'break'.
To reproduce, set up a test following the snippets below, and upload a
file named 'breakhere.jpg' to the resulting form.
I'm using Apache/2.2.9 (Ubuntu) DAV/2 SVN/1.5.1 mod_ssl/2.2.9
OpenSSL/0.9.8g mod_apreq2-20051231/2.6.0 mod_perl/2.0.4 Perl/v5.10.0
In your error log you will see just 'break'.
---httpd.conf---
Perl
use lib qw(/home/mcrawfor/);
/Perl
PerlModule QuoteParse
Location /test/
SetHandler perl-script
PerlResponseHandler QuoteParse
/Location
---QuoteParse.pm---
package QuoteParse;
use strict;
use warnings;
use Apache2::RequestRec ();
use Apache2::Request ();
use Apache2::Const -compile = qw(OK);
sub handler {
my $r = shift;
my $req = Apache2::Request-new($r);
warn $req-param('foo');
$r-content_type('text/html');
print form method='post' enctype='multipart/form-data'input
type='file' name='foo'input type='submit'/form;
return Apache2::Const::OK;
}
1;
---End Message---
---BeginMessage---
Miles Crawford wrote:
When you handle a multipart/form-data post with libapreq quotes in
filenames are mishandled. For example, a post that includes:
Content-Disposition: form-data; name=foo; filename=breakhere.jpg
Isn't that a malformed header? I would think that the internal ''
should be escaped for it to be properly represent that filename.
Assuming that's the case, the rest of the behavior that you describe is
what I'd expect.
Adam
---End Message---
---BeginMessage---
I agree it looks bogus, but safari and firefox send the header in that
format. Not sure about IE since Windows does not allow files with a
in the name.
I looked over the Content-Disposition header RFC but it does not seem
to address escaping directly.
Either way, it seems that this format is a browser fact of life
-miles
On Fri, Jan 2, 2009 at 4:44 PM, Adam Prime adam.pr...@utoronto.ca wrote:
Miles Crawford wrote:
When you handle a multipart/form-data post with libapreq quotes in
filenames are mishandled. For example, a post that includes:
Content-Disposition: form-data; name=foo; filename=breakhere.jpg
Isn't that a malformed header? I would think that the internal '' should
be escaped for it to be properly represent that filename. Assuming that's
the case, the rest of the behavior that you describe is what I'd expect.
Adam
---End Message---
Where to initialize a global pool/hash - server create or child_init?
Where to begin I am creating a global pool/hash in which I save cached, hard to calculate data (pre-compiled regex expressions etc.). I also store pages I've created using this data in memcached (using the APR interface). I can't save the calculated regex data in memcached. I initially followed the advice of creating a private pool, hash and mutex inside a child_init hook in my server config structure, protecting all access with the mutex. The module works well enough in regular httpd. However, when I tried this in worker MPM I got constant: [notice] child pid x exit signal Segmentation fault (11) after a few page loads. On a whim, I moved the whole creation of these structures into a server config hook. All these issues seem to have vanished. I have theorized that the pool I am creating at every child creation does not properly work. I am looking for a discussion of the pros and cons of creating this in a per child hook, versus the one time server create hook and any other pointers to help decide (and debug) where/when I should use one or the other. Boy, I hope this makes sense.
Re: [VOTE] Release Apache HTTP server 2.3.1-alpha
On 01/03/2009 07:39 AM, Paul Querna wrote: Test tarballs for Apache httpd 2.3.1-alpha are available at: http://httpd.apache.org/dev/dist/ Your votes please; ±1 [ ] Release httpd-2.3.0 as Alpha Vote closes at 7:00 UTC on Thursday January 8 2009. -1. It crashes on start for me: #0 0xb7e67140 in apr_table_get (t=0x0, key=0x80a6d40 http) at tables/apr_tables.c:475 #1 0x0808042f in ap_apply_accept_filter (p=0x80a50a0, lis=value optimized out, server=0x82f9bf8) at listen.c:190 #2 0x08080c13 in ap_setup_listeners (s=0x80acb00) at listen.c:569 #3 0x0808bf2a in worker_open_logs (p=0x80a70a8, plog=0x80d51f0, ptemp=0x80d71f8, s=0x80acb00) at worker.c:1888 #4 0x08078630 in ap_run_open_logs (pconf=0x80a70a8, plog=0x80d51f0, ptemp=0x80d71f8, s=0x80acb00) at config.c:152 #5 0x08064b0d in main (argc=134893728, argv=0x837ae20) at main.c:709 This is fixed in r730949. Regards Rüdiger
Re: svn commit: r730882 - /httpd/httpd/trunk/configure.in
On 03.01.2009 08:52, Justin Erenkrantz wrote: [ It's odd as I didn't get the email for this commit...anyway... ] Author: rjung Date: Fri Jan 2 17:01:56 2009 New Revision: 730882 URL: http://svn.apache.org/viewvc?rev=730882view=rev Log: Only link libhttpd against pcre. Modified: httpd/httpd/trunk/configure.in Modified: httpd/httpd/trunk/configure.in URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/configure.in?rev=730882r1=730881r2=730882view=diff == --- httpd/httpd/trunk/configure.in (original) +++ httpd/httpd/trunk/configure.in Fri Jan 2 17:01:56 2009 @@ -182,7 +182,8 @@ fi AC_MSG_NOTICE([Using external PCRE library from $PCRE_CONFIG]) APR_ADDTO(CFLAGS, [`$PCRE_CONFIG --cflags`]) - APR_ADDTO(LIBS, [`$PCRE_CONFIG --libs`]) + APR_ADDTO(LTLIBRARY_LIBADD, [`$PCRE_CONFIG --libs`]) + APACHE_SUBST(LTLIBRARY_LIBADD) else AC_MSG_ERROR([pcre-config for libpcre not found. PCRE is required and available from http://pcre.org/]) fi This change busts my build because it tries to link a dynamic library (pcre) against a static library (libmain). You can't do that. (jlibtool barfs on this.) It is only legal to link static libraries against other static libraries. Please revert. -- justin There was no problem with usual libtool. I think it simply records the dependency in the *.la file (but didn't check that). Does the following change fix it for MacOS? Index: configure.in === --- configure.in(revision 730882) +++ configure.in(working copy) @@ -182,8 +182,7 @@ fi AC_MSG_NOTICE([Using external PCRE library from $PCRE_CONFIG]) APR_ADDTO(CFLAGS, [`$PCRE_CONFIG --cflags`]) - APR_ADDTO(LTLIBRARY_LIBADD, [`$PCRE_CONFIG --libs`]) - APACHE_SUBST(LTLIBRARY_LIBADD) + APR_ADDTO(HTTPD_LDFLAGS, [`$PCRE_CONFIG --libs`]) else AC_MSG_ERROR([pcre-config for libpcre not found. PCRE is required and available from http://pcre.org/]) fi If so, that's better for several reasons. Regards, Rainer
Re: svn commit: r730835 - /httpd/httpd/trunk/server/core.c
On 01/02/2009 10:50 PM, Ruediger Pluem wrote:
On 01/02/2009 10:27 PM, William A. Rowe, Jr. wrote:
Ruediger Pluem wrote:
Otherwise the table will not be empty anymore in the case that
neither APR_HAS_SO_ACCEPTFILTER nor APR_TCP_DEFER_ACCEPT is set.
Uhm ... huh? What gave you the idea that APR_TCP_DEFER_ACCEPT
is a volatile? It's present in apr 1.3 (our baseline) and will
be sticking around into the foreseeable future.
It is neither a HAS nor HAVE feature flag. In fact it's the reason
I started refactoring this code, it was complete twaddle.
As I dig deeper into this I think that the whole AcceptFilter config
is busted:
ap_apply_accept_filter allows you set up individual accept filters
for each listening socket. This seems right to me.
OTOH AcceptFilter only allows you to do a global mapping of protocols
to accept filters.
Thus if I want to have two different listeners with the same protocol
but different accept filters I cannot do this configuration wise.
This doesn't seem correct to me.
Furthermore if all this stuff is global the following loop from
listen.c::ap_setup_listeners doesn't really make sense to me:
for (lr = ap_listeners; lr; lr = lr-next) {
num_listeners++;
found = 0;
for (ls = s; ls !found; ls = ls-next) {
for (addr = ls-addrs; addr !found; addr = addr-next) {
if (apr_sockaddr_equal(lr-bind_addr, addr-host_addr)
lr-bind_addr-port == addr-host_port) {
found = 1;
ap_apply_accept_filter(s-process-pool, lr, ls);
}
}
}
if (!found) {
ap_apply_accept_filter(s-process-pool, lr, s);
}
}
Why do we need to iterate over the server recs then (2nd loop)?
The config information about protocol filter mappings should be
only in the main server rec.
Regards
Rüdiger
Re: Undefined symbols in ab.c
Rainer Jung wrote: It works for me on Solaris. Those symbols (without the leading underscore) are referenced indeed by ab.c, but they should be in your libcrypto (BIO*) resp. libssl (SSL*). Are you sure, that the libraries libcrypto and libssl can be found? Are they in /Users/minfrin/src/apache/sandbox/crypto/nss-3.12/mozilla/dist/Darwin9.4.0_OPT.OBJ/lib? Which version of OpenSSL do you use (I use 0.9.8i)? Can you see the symbols in the libs (check with nm)? I had checked with nm, and the symbols weren't present in either libssl or libcrypto v0.9.7 (as shipped with MacOSX v10.5), or v0.9.8 (from Macports), nor in openssl-0.9.8b (from RHEL5). Busy trying to build trunk from a pristine copy, will see if it makes a difference. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
httpd build variables
Most build variables for httpd are used via APACHE_SUBST, which means they get added to build/config_vars.mk. A) Where to define them? There are two places, were a lot of ariables are added via APACHE_SUBST: 1) Macro APACHE_GEN_CONFIG_VARS Defined in acinclude.m4 and called in configure.in. It contains 75 calls to APACHE_SUBST. 2) Directly in configure.in It contains about 21 calls to APACHE_SUBST. Is there any concept, whether APACHE_SUBST for a new variable should be put into APACHE_GEN_CONFIG_VARS or directly into configure.in? I can see that the ones in the macro will be included last into config_vars.mk, but apart from that any rules? B) Should they be installed? The script build/config_vars.sh deletes a lot of variables before installing build/config_vars.mk. Is there any concept, which variables should get installed, and which ones should be removed before installing the file? Regards, Rainer
Re: svn commit: r730717 - /httpd/httpd/trunk/modules/session/config.m4
rj...@apache.org wrote: URL: http://svn.apache.org/viewvc?rev=730717view=rev Log: Add a header check for apr_ssl.h to mod_session_crypto. The modules needs the header which is at the moment only part of the ssl-evp branch of APR. I have just finished updating session_crypto to depend on the apr_crypto.h API instead of the deprecated apr_ssl.h API, however the check below, once changed to apr_crypto.h, doesn't seem to detect the header properly. Can you verify whether this works for you against apr_crypto.h? Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: svn commit: r730717 - /httpd/httpd/trunk/modules/session/config.m4
On 03.01.2009 16:02, Graham Leggett wrote:
rj...@apache.org wrote:
URL: http://svn.apache.org/viewvc?rev=730717view=rev
Log:
Add a header check for apr_ssl.h to mod_session_crypto.
The modules needs the header which is at the moment
only part of the ssl-evp branch of APR.
I have just finished updating session_crypto to depend on the
apr_crypto.h API instead of the deprecated apr_ssl.h API, however the
check below, once changed to apr_crypto.h, doesn't seem to detect the
header properly.
Can you verify whether this works for you against apr_crypto.h?
I see, I forgot to add the APR and APU include paths to CPPFLAGS before
testing the header. Autoconf does not only look for the file, it also
processes it via CPP and compiles it.
I tested it now with apr_ssl.h and apr_crypto.h. The following change
should do it for you:
1) Cleanup of apu_errno.h
-
--- apu_errno.h 2009-01-03 16:52:53.0 +0100
+++ apu_errno.h 2009-01-03 16:49:53.0 +0100
@@ -22,8 +22,8 @@
* @brief APR-Util Error Codes
*/
-#include apr.h
-#include apr_errno.h
+#include apr.h
+#include apr_errno.h
#ifdef __cplusplus
extern C {
2) Changing the feature test
--- modules/session/config.m4 (revision 730878)
+++ modules/session/config.m4 (working copy)
@@ -18,8 +18,11 @@
APACHE_MODULE(session, session module, , , most)
APACHE_MODULE(session_cookie, session cookie module, , ,
$session_mods_enable)
APACHE_MODULE(session_crypto, session crypto module, , , no, [
- AC_CHECK_HEADERS(apr_ssl.h, [ap_HAVE_APR_SSL_H=yes],
[ap_HAVE_APR_SSL_H=no])
- if test $ap_HAVE_APR_SSL_H = no; then
+ saved_CPPFLAGS=$CPPFLAGS
+ CPPFLAGS=$CPPFLAGS -I$APR_INCLUDEDIR -I$APU_INCLUDEDIR
+ AC_CHECK_HEADERS(apr_crypto.h, [ap_HAVE_APR_CRYPTO=yes],
[ap_HAVE_APR_CRYPTO=no])
+ CPPFLAGS=$saved_CPPFLAGS
+ if test $ap_HAVE_APR_CRYPTO = no; then
AC_MSG_WARN([Your APR does not include SSL/EVP support.])
enable_session_crypto=no
fi
As you can see I switched to apr_crypto.h, but also added paths
temporarily to CPPFLAGS.
3) Resulting changes to configure
-
The resulting changes to configure are:
--- configure.kpdt_orig 2009-01-03 13:59:32.0 +0100
+++ configure 2009-01-03 16:58:24.0 +0100
@@ -18946,8 +18946,10 @@
{ echo $as_me:$LINENO: result: checking dependencies 5
echo ${ECHO_T}checking dependencies 6; }
+ saved_CPPFLAGS=$CPPFLAGS
+ CPPFLAGS=$CPPFLAGS -I$APR_INCLUDEDIR -I$APU_INCLUDEDIR
-for ac_header in apr_ssl.h
+for ac_header in apr_crypto.h
do
as_ac_Header=`echo ac_cv_header_$ac_header | $as_tr_sh`
if { as_var=$as_ac_Header; eval test \\${$as_var+set}\ = set; }; then
@@ -19081,14 +19083,15 @@
cat confdefs.h _ACEOF
#define `echo HAVE_$ac_header | $as_tr_cpp` 1
_ACEOF
- ap_HAVE_APR_SSL_H=yes
+ ap_HAVE_APR_CRYPTO=yes
else
- ap_HAVE_APR_SSL_H=no
+ ap_HAVE_APR_CRYPTO=no
fi
done
- if test $ap_HAVE_APR_SSL_H = no; then
+ CPPFLAGS=$saved_CPPFLAGS
+ if test $ap_HAVE_APR_CRYPTO = no; then
{ echo $as_me:$LINENO: WARNING: Your APR does not include SSL/EVP
support. 5
echo $as_me: WARNING: Your APR does not include SSL/EVP support. 2;}
enable_session_crypto=no
If the feature test worked for your with apr_ssl.h, then it's possible,
that you can fix it with 1). I will apply 2), because that's cleaner,
but you might nevertheless add 1) to apr-util.
If it doesn't work for you, look inside config.log. You'll find some
more info there, which part of the test failed, and why.
Regards,
Rainer
Re: svn commit: r730984 - /httpd/httpd/trunk/configure.in
On Sat, Jan 3, 2009 at 5:37 AM, rj...@apache.org wrote: Author: rjung Date: Sat Jan 3 05:37:05 2009 New Revision: 730984 URL: http://svn.apache.org/viewvc?rev=730984view=rev Log: Fix correct linking against PCRE. r730882 broke builds on MacOSX. Works fine now on Mac OS X w/jlibtool. Thanks. -- justin
Re: svn commit: r730949 - /httpd/httpd/trunk/server/listen.c
rpl...@apache.org wrote:
Author: rpluem
Date: Sat Jan 3 01:52:12 2009
New Revision: 730949
URL: http://svn.apache.org/viewvc?rev=730949view=rev
Log:
* Handle the case where conf-accf_map is NULL gracefully instead of
segfaulting
Modified:
httpd/httpd/trunk/server/listen.c
Modified: httpd/httpd/trunk/server/listen.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/server/listen.c?rev=730949r1=730948r2=730949view=diff
==
--- httpd/httpd/trunk/server/listen.c (original)
+++ httpd/httpd/trunk/server/listen.c Sat Jan 3 01:52:12 2009
@@ -183,7 +183,7 @@
const char* accf;
core_server_config *conf = ap_get_module_config(s-module_config,
core_module);
-if (!proto) {
+if (!proto || !conf-accf_map) {
Ok, problem there. AIUI this should not happen.
accf_map is created for the global-hosts (!is_virtual). All virtual hosts
should have run the merge, which cpy's the global before fixing up specific
fields in the merge.
I'd prefer we revert and work backwards to where the bug really is.
Re: svn commit: r730835 - /httpd/httpd/trunk/server/core.c
Ruediger Pluem wrote: As I dig deeper into this I think that the whole AcceptFilter config is busted: Ahhh, better explanation, thanks! Note AcceptFilter is ONLY accepted in the global context, so all that logic below was nonsense. ap_apply_accept_filter allows you set up individual accept filters for each listening socket. This seems right to me. OTOH AcceptFilter only allows you to do a global mapping of protocols to accept filters. Thus if I want to have two different listeners with the same protocol but different accept filters I cannot do this configuration wise. This doesn't seem correct to me. Furthermore if all this stuff is global the following loop from listen.c::ap_setup_listeners doesn't really make sense to me: You can, as long as they are named by different protocols. There's really not a logical reason for extra flexibility by protocol. If one host exists only to deal with some broken client, http_borked protocol name for that IP would be fine. Why do we need to iterate over the server recs then (2nd loop)? The config information about protocol filter mappings should be only in the main server rec. Sounds right.
Re: svn commit: r730949 - /httpd/httpd/trunk/server/listen.c
William A. Rowe, Jr. wrote:
rpl...@apache.org wrote:
Author: rpluem
Date: Sat Jan 3 01:52:12 2009
New Revision: 730949
URL: http://svn.apache.org/viewvc?rev=730949view=rev
Log:
* Handle the case where conf-accf_map is NULL gracefully instead of
segfaulting
Modified:
httpd/httpd/trunk/server/listen.c
Modified: httpd/httpd/trunk/server/listen.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/server/listen.c?rev=730949r1=730948r2=730949view=diff
==
--- httpd/httpd/trunk/server/listen.c (original)
+++ httpd/httpd/trunk/server/listen.c Sat Jan 3 01:52:12 2009
@@ -183,7 +183,7 @@
const char* accf;
core_server_config *conf = ap_get_module_config(s-module_config,
core_module);
-if (!proto) {
+if (!proto || !conf-accf_map) {
Ok, problem there. AIUI this should not happen.
accf_map is created for the global-hosts (!is_virtual). All virtual hosts
should have run the merge, which cpy's the global before fixing up specific
fields in the merge.
I'd prefer we revert and work backwards to where the bug really is.
I'm still confused, in merge_core_server_configs, we do;
conf = (core_server_config *)apr_pmemdup(p, virt, sizeof(core_server_config));
and otherwise didn't touch the accf_map pointer. How do you have an orphan
virtual server config which was never merged to the core server config?!?
Are you saying we actually iterate UNMERGED server confs in places? Now
that is horridly wrong behavior.
Re: svn commit: r730949 - /httpd/httpd/trunk/server/listen.c
William A. Rowe, Jr. wrote: conf = (core_server_config *)apr_pmemdup(p, virt, sizeof(core_server_config)); Bloody heck, this is ridiculous. The error is right there, you *must* always dup the base and then apply non-defaults from virt. Working on it.
Re: svn commit: r731009 - /httpd/httpd/trunk/modules/session/config.m4
On 01/03/2009 05:12 PM, rj...@apache.org wrote: Author: rjung Date: Sat Jan 3 08:12:55 2009 New Revision: 731009 URL: http://svn.apache.org/viewvc?rev=731009view=rev Log: Allow feature test for apr_ssl.h to succeed: Add necessary search paths temporarily to CPPFLAGS. Addition to r730717. When switching to another header, no need to again rename the ap_HAVE_APR_CRYPTO variable. Modified: httpd/httpd/trunk/modules/session/config.m4 Modified: httpd/httpd/trunk/modules/session/config.m4 URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/session/config.m4?rev=731009r1=731008r2=731009view=diff == --- httpd/httpd/trunk/modules/session/config.m4 (original) +++ httpd/httpd/trunk/modules/session/config.m4 Sat Jan 3 08:12:55 2009 @@ -18,8 +18,11 @@ APACHE_MODULE(session, session module, , , most) APACHE_MODULE(session_cookie, session cookie module, , , $session_mods_enable) APACHE_MODULE(session_crypto, session crypto module, , , no, [ - AC_CHECK_HEADERS(apr_ssl.h, [ap_HAVE_APR_SSL_H=yes], [ap_HAVE_APR_SSL_H=no]) - if test $ap_HAVE_APR_SSL_H = no; then + saved_CPPFLAGS=$CPPFLAGS Does it make sense to save these flags if you do not restore them later on? + CPPFLAGS=$CPPFLAGS -I$APR_INCLUDEDIR -I$APU_INCLUDEDIR + AC_CHECK_HEADERS(apr_crypto.h, [ap_HAVE_APR_CRYPTO=yes], [ap_HAVE_APR_CRYPTO=no]) + CPPFLAGS=$saved_CPPFLAGS + if test $ap_HAVE_APR_CRYPTO = no; then AC_MSG_WARN([Your APR does not include SSL/EVP support.]) enable_session_crypto=no fi Regards Rüdiger
Re: svn commit: r731009 - /httpd/httpd/trunk/modules/session/config.m4
On 01/04/2009 12:01 AM, Ruediger Pluem wrote: On 01/03/2009 05:12 PM, rj...@apache.org wrote: Author: rjung Date: Sat Jan 3 08:12:55 2009 New Revision: 731009 URL: http://svn.apache.org/viewvc?rev=731009view=rev Log: Allow feature test for apr_ssl.h to succeed: Add necessary search paths temporarily to CPPFLAGS. Addition to r730717. When switching to another header, no need to again rename the ap_HAVE_APR_CRYPTO variable. Modified: httpd/httpd/trunk/modules/session/config.m4 Modified: httpd/httpd/trunk/modules/session/config.m4 URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/session/config.m4?rev=731009r1=731008r2=731009view=diff == --- httpd/httpd/trunk/modules/session/config.m4 (original) +++ httpd/httpd/trunk/modules/session/config.m4 Sat Jan 3 08:12:55 2009 @@ -18,8 +18,11 @@ APACHE_MODULE(session, session module, , , most) APACHE_MODULE(session_cookie, session cookie module, , , $session_mods_enable) APACHE_MODULE(session_crypto, session crypto module, , , no, [ - AC_CHECK_HEADERS(apr_ssl.h, [ap_HAVE_APR_SSL_H=yes], [ap_HAVE_APR_SSL_H=no]) - if test $ap_HAVE_APR_SSL_H = no; then + saved_CPPFLAGS=$CPPFLAGS Does it make sense to save these flags if you do not restore them later on? + CPPFLAGS=$CPPFLAGS -I$APR_INCLUDEDIR -I$APU_INCLUDEDIR + AC_CHECK_HEADERS(apr_crypto.h, [ap_HAVE_APR_CRYPTO=yes], [ap_HAVE_APR_CRYPTO=no]) + CPPFLAGS=$saved_CPPFLAGS Ahh, sorry. Out of coffee error. Regards Rüdiger
Problem with file descriptor handling in httpd 2.3.1
During testing 2.3.1 I noticed a lot of errors of type EMFILE: Too many open files. I used strace and the problem looks like this: - The test case is using ab with HTTP keep alive, concurrency 20 and a small file, so doing about 2000 requests per second. MaxKeepAliveRequests=100 (Default) - the file leading to EMFILE is the static content file, which can be observed to be open more than 1000 times in parallel although ab concurrency is only 20 - From looking at the code it seems the file is closed during a cleanup function associated to the request pool, which is triggered by an EOR bucket Now what happens under KeepAlive is that the content files are kept open longer than the handling of the request, more precisely until the closing of the connection. So when MaxKeepAliveRequests*Concurrency MaxNumberOfFDs we run out of file descriptors. I observed the behaviour with 2.3.1 on Linux (SLES10 64Bit) with Event, Worker and Prefork. I didn't yet have the time to retest with 2.2. For Event and Worker I get also crashes (more precisely httpd processes stopping) due to apr_socket_accept() also returning with EMFILE. Regards, Rainer
Re: Problem with file descriptor handling in httpd 2.3.1
Rainer Jung wrote: During testing 2.3.1 I noticed a lot of errors of type EMFILE: Too many open files. I used strace and the problem looks like this: - The test case is using ab with HTTP keep alive, concurrency 20 and a small file, so doing about 2000 requests per second. MaxKeepAliveRequests=100 (Default) - the file leading to EMFILE is the static content file, which can be observed to be open more than 1000 times in parallel although ab concurrency is only 20 - From looking at the code it seems the file is closed during a cleanup function associated to the request pool, which is triggered by an EOR bucket Now what happens under KeepAlive is that the content files are kept open longer than the handling of the request, more precisely until the closing of the connection. So when MaxKeepAliveRequests*Concurrency MaxNumberOfFDs we run out of file descriptors. I observed the behaviour with 2.3.1 on Linux (SLES10 64Bit) with Event, Worker and Prefork. I didn't yet have the time to retest with 2.2. It should only happen in 2.3.x/trunk because the EOR bucket is a new feature to let MPMs do async writes once the handler has finished running. And yes, this sounds like a nasty bug. -Paul
Re: Problem with file descriptor handling in httpd 2.3.1
On 04.01.2009 00:36, Paul Querna wrote: Rainer Jung wrote: During testing 2.3.1 I noticed a lot of errors of type EMFILE: Too many open files. I used strace and the problem looks like this: - The test case is using ab with HTTP keep alive, concurrency 20 and a small file, so doing about 2000 requests per second. MaxKeepAliveRequests=100 (Default) - the file leading to EMFILE is the static content file, which can be observed to be open more than 1000 times in parallel although ab concurrency is only 20 - From looking at the code it seems the file is closed during a cleanup function associated to the request pool, which is triggered by an EOR bucket Now what happens under KeepAlive is that the content files are kept open longer than the handling of the request, more precisely until the closing of the connection. So when MaxKeepAliveRequests*Concurrency MaxNumberOfFDs we run out of file descriptors. I observed the behaviour with 2.3.1 on Linux (SLES10 64Bit) with Event, Worker and Prefork. I didn't yet have the time to retest with 2.2. It should only happen in 2.3.x/trunk because the EOR bucket is a new feature to let MPMs do async writes once the handler has finished running. And yes, this sounds like a nasty bug. I verified I can't reproduce with the same platform and 2.2.11. Not sure I understand the EOR asynchronicity good enough to analyze the root cause. Rainer
Re: Problem with file descriptor handling in httpd 2.3.1
On 01/04/2009 12:49 AM, Rainer Jung wrote: On 04.01.2009 00:36, Paul Querna wrote: Rainer Jung wrote: During testing 2.3.1 I noticed a lot of errors of type EMFILE: Too many open files. I used strace and the problem looks like this: - The test case is using ab with HTTP keep alive, concurrency 20 and a small file, so doing about 2000 requests per second. What is the exact size of the file? Regards Rüdiger
