Re: [pre-release] 2.0.55 *candidate* available for testing
Tested successfully on Linux 2.6.13/x86_64 (Fedora Core 4) with both worker and prefork MPMs. I encountered lots of errors in perl-framework's t/TEST with prefork on Darwin 8.2.0/PPC (OS X 10.4.2). I don't yet know whether these are due to httpd-2.0.55 problems or just problems with my Perl installation. Brian On Oct 9, 2005, at 9:42 PM, William A. Rowe, Jr. wrote: The httpd-2.0.55 candidate, including win32 source .zip and installers*, is now available for testing at http://httpd.apache.org/dev/dist/ Please review this candidate, and when responding, indicate the precise operating system that you have tested. Thank you for your assistance! Bill * note that win32 binary installers were uploaded only at this hour, and it will take up to another two hours for the public server to resync. Thanks for your patience.
async write completion prototype
With the batch of commits I did this weekend, the Event MPM in the async-dev Subversion branch now does write completion in a nonblocking manner. Once an entire response has been generated and passed to the output filter chain, the MPM's poller/listener thread watches the connection for writability events. When the connection becomes writable, the poller thread sends it to one of the worker threads, which writes some more output. At this point, the event-handling code is ready for testing and review by other developers. The main changes on the async-dev branch (compared to the 2.3-dev trunk) are: 1. ap_core_output_filter: rewrite to do nonblocking writes whenever possible. 2. core, http module, and mod_logio: removed the generation of flush buckets where possible. 3. request cleanup and logging: the logger phase and subsequent destruction of the request's pool are now triggered by the destruction of an End-Of-Request bucket in the core output filter. 4. event MPM: asynchronous handling of CONN_STATE_WRITE_COMPLETION. There are several more things that need to be fixed in order to make the asynchronous write completion useful in a production release of httpd-2.x: - The main pollset in the Event MPM currently is sized to hold up to one socket descriptor per worker thread. With asynchronous keepalives and write completion, the pollset should accommodate many descriptors per thread. - The logic for starting more child processes, which Event inherited from Worker, is based on assumptions about the number of concurrent connections being equal to the number of threads. These assumptions aren't valid for a multiple-connections-per-thread MPM. - Similarly, there may be some changes needed in the flow control logic that the listener thread uses to decide whether it can do an accept. - The scoreboard probably needs a redesign. - It may be valuable to have a separate thread pool to run handlers that do arbitrarily lengthy processing, such as mod_perl and mod_php. Brian
Re: async write completion prototype
Brian Pane wrote: With the batch of commits I did this weekend, the Event MPM in the async-dev Subversion branch now does write completion in a nonblocking manner. Once an entire response has been generated and passed to the output filter chain, the MPM's poller/listener thread watches the connection for writability events. When the connection becomes writable, the poller thread sends it to one of the worker threads, which writes some more output. If the content has already been generated, why add the overhead of the context switch/sending to another thread? Can't the same event thread do a non-blocking write? Once it finishes writing, then yes, we do require a context-switch to another thread to do logging/cleanup. I am mostly thinking about downloading a 1 gig file with the current pattern against a slow client. A non-blocking write might only do ~64k at a time, and causing 1 gig/64k context switches, which seems less than optimal. ... - The main pollset in the Event MPM currently is sized to hold up to one socket descriptor per worker thread. With asynchronous keepalives and write completion, the pollset should accommodate many descriptors per thread. The pollset is auto-resizable. That number is just the maximum number of events that will ever be returned by a single call to _poll(). This number if perfect for the number of threads, since we can never dispatch to more than the number of threads we have... - The scoreboard probably needs a redesign. Yes.. it is completely unhelpful with the maximum number of connections being number of threads. I suspect going forward there will be many other areas where this assumption is broken by the event mpm. -Paul
Re: [pre-release] 2.0.55 *candidate* available for testing
måndag 10 oktober 2005 06.42 skrev William A. Rowe, Jr.: The httpd-2.0.55 candidate, including win32 source .zip and installers*, is now available for testing at http://httpd.apache.org/dev/dist/ Please review this candidate, and when responding, indicate the precise operating system that you have tested. There's no fix for CAN-2005-2088 in this one. -- Regards // Oden Eriksson Mandriva: http://www.mandriva.com NUX: http://nux.se
Re: [pre-release] 2.0.55 *candidate* available for testing
måndag 10 oktober 2005 09.54 skrev Oden Eriksson: måndag 10 oktober 2005 06.42 skrev William A. Rowe, Jr.: The httpd-2.0.55 candidate, including win32 source .zip and installers*, is now available for testing at http://httpd.apache.org/dev/dist/ Please review this candidate, and when responding, indicate the precise operating system that you have tested. There's no fix for CAN-2005-2088 in this one. Duh! Too early in the morning... -- Regards // Oden Eriksson Mandriva: http://www.mandriva.com NUX: http://nux.se
Re: [pre-release] 2.0.55 *candidate* available for testing
Brian Pane said: I encountered lots of errors in perl-framework's t/TEST with prefork on Darwin 8.2.0/PPC (OS X 10.4.2). I don't yet know whether these are due to httpd-2.0.55 problems or just problems with my Perl installation. I ran the build/binbuild.sh script, and httpd built clean on my Darwin 8.2.0. I didn't see any tests being run though, is the test suite fired off from the binbuild script? Regards, Graham --
Re: svn commit: r220307 - in /httpd/httpd/trunk/modules: metadata/mod_setenvif.c ssl/mod_ssl.c ssl/mod_ssl.h ssl/ssl_expr_eval.c
David Reid wrote: Joe Orton wrote: On Fri, Aug 05, 2005 at 08:00:01PM +0200, Martin Kraemer wrote: On Tue, Aug 02, 2005 at 07:14:10PM +0200, Martin Kraemer wrote: I wanted something like SSLRequire committers in SSLPeerExtList(1.3.6.1.4.1.18060.1); to mean at least one extension with an OID of 1.3.6.1.4.1.18060.1 with a value of 'committers' exists in the client cert. I'll be on vacation next week, and will send in another patch after that. OK, hope you had a good holiday. I wasn't trying to argue about the semantics just to nitpick the naming. Having SSL in the SSLRequire function is redundant, but not in the context of mod_setenvif. So, my preference is: SSLRequire committers in PeerExtList(1.3.6.1.4.1.18060.1); SetEnvIf SSL_PeerExtList(etc) ... +1 on the revised naming. Patch looks OK as well. Late to the party here, but giving the same function two different names sucks! Or am I misunderstanding? Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff
Re: [pre-release] 2.0.55 *candidate* available for testing
måndag 10 oktober 2005 06.42 skrev William A. Rowe, Jr.: The httpd-2.0.55 candidate, including win32 source .zip and installers*, is now available for testing at http://httpd.apache.org/dev/dist/ Please review this candidate, and when responding, indicate the precise operating system that you have tested. Thank you for your assistance! Bill * note that win32 binary installers were uploaded only at this hour, and it will take up to another two hours for the public server to resync. Thanks for your patience. I saw this: Cannot load /etc/httpd/modules/mod_cgi.so into server: /etc/httpd/modules/mod_cgi.so: undefined symbol: apr_procattr_addrspace_set And some investigations told me it requires apr 0.9.7, maybe the autotools stuff should check for this or be documented? -- Regards // Oden Eriksson Mandriva: http://www.mandriva.com NUX: http://nux.se
Re: AP_INIT_* macros in httpd-2.1
On Sat, Oct 08, 2005 at 08:52:23PM +0100, Nick Kew wrote: I've just stumbled on something perplexing. Compiling mod_validator[1], a large C++ module, for 2.1, I found each entry in the command_rec generating a syntax error. They are fine with 2.0.x. AP_HAVE_DESIGNATED_INITIALIZER is defined in ap_config.h in 2.1.x rather than in CPPFLAGS per 2.0.x, but I guess this is a reason why not to do that. Alternatively I guess ap_config.h could do #ifdef __cplusplus #undef AP_HAVE_DESIGNATED_INITIALIZER #endif if this is simply not kosher C++ syntax. joe
MMN and branches (was Re: svn commit: r307031)
On Sat, Oct 08, 2005 at 04:51:51PM -0500, William Rowe wrote: Joe Orton wrote: I'd suggest just incrementing the MMN by one rather than bumping it by date on the branch unless and until the API is the same as the trunk. I previously considered that. Although you are right; they are out of sync, I think it's inevitable with two branches following different history, no? It's not inevitable but it is the status quo ;) This is why the MODULE_MAGIC_COOKIE is now AP24 on svn head, because the two branches can't remain binary compatible, at least not except for some very short periods of time. Perhaps you are right, perhaps it's time to drop date stamps as the MODULE_MAGIC_NUMBER_MAJOR signature? Maintaining an MMN which is intended to reflect every single ABI change and multiple branches with independent ABIs is going to be a tricky problem. Alternatives I can see: a) partition the MMN major by MODULE_MAGIC_COOKIE, so modules must test an (MODULE_MAGIC_COOKIE, MODULE_MAGIC_NUMBER_MAJOR) pair to target a particular API - results are then reliable across history and across branches - icky b) stop bumping the MMN by date on branches, just increment by one and hope that enough gap is left - results are then reliable across branches but not across history c) don't backport ABI changes to branches; a branch much fork from a specific MODULE_MAGIC_NUMBER_MAJOR and must be rebased entirely to move to a new major - avoids the problem entirely d) drop the requirement that the MMN must reflect every single ABI change and make some more fundamental change joe
Re: [pre-release] 2.0.55 *candidate* available for testing
On Oct 10, 2005, at 2:22 AM, Graham Leggett wrote: Brian Pane said: I encountered lots of errors in perl-framework's t/TEST with prefork on Darwin 8.2.0/PPC (OS X 10.4.2). I don't yet know whether these are due to httpd-2.0.55 problems or just problems with my Perl installation. I ran the build/binbuild.sh script, and httpd built clean on my Darwin 8.2.0. I didn't see any tests being run though, is the test suite fired off from the binbuild script? The test suite has to be downloaded and run separately. Basically, - do a make install of the httpd - checkout asf/repos/httpd/test/trunk from SVN - cd to the perl-framework subdirectory - perl Makefile.pl -apxs /path/to/the/httpd/installation/bin/apxs - ./t/TEST Brian
Re: MMN and branches (was Re: svn commit: r307031)
Joe Orton wrote: c) don't backport ABI changes to branches; a branch much fork from a specific MODULE_MAGIC_NUMBER_MAJOR and must be rebased entirely to move to a new major - avoids the problem entirely It's always this way on even branches, but we are talking about odd #'ed branches, here. d) drop the requirement that the MMN must reflect every single ABI change and make some more fundamental change I've always considered the possibility that we only bump MMN major, by date, exactly once when we roll rev x.y.n shrug. Better idea; let's refactor the whole MMN magic in httpd 3.0? (It's already looking, from the scope of the experimental asynch branch, that it's likely to become 3.0.) Bill
Re: [pre-release] 2.0.55 *candidate* available for testing
Oden Eriksson wrote: And some investigations told me it requires apr 0.9.7, maybe the autotools stuff should check for this or be documented? Yup - apr 0.9.7 is part of the bundle. We can spell this out in the announce, certainly, and on the downloads page README - would that suffice?
Re: [pre-release] 2.0.55 *candidate* available for testing
Brian Pane wrote: I encountered lots of errors in perl-framework's t/TEST with prefork on Darwin 8.2.0/PPC (OS X 10.4.2). I don't yet know whether these are due to httpd-2.0.55 problems or just problems with my Perl installation. Hmmm... review this bug (not fixed in 0.9.7, afaict); http://issues.apache.org/bugzilla/show_bug.cgi?id=34332 Bill
Re: [pre-release] 2.0.55 *candidate* available for testing
måndag 10 oktober 2005 16.27 skrev William A. Rowe, Jr.: Oden Eriksson wrote: And some investigations told me it requires apr 0.9.7, maybe the autotools stuff should check for this or be documented? Yup - apr 0.9.7 is part of the bundle. We can spell this out in the announce, certainly, and on the downloads page README - would that suffice? Yep. Thanks. -- Regards // Oden Eriksson Mandriva: http://www.mandriva.com NUX: http://nux.se
Re: [pre-release] 2.0.55 *candidate* available for testing
William A. Rowe, Jr. wrote: Yup - apr 0.9.7 is part of the bundle. We can spell this out in the announce, certainly, and on the downloads page README - would that suffice? How horrible would it be to have the apr_reslist_invalidate patch applied to the bundled apr? ducks and runs / -- Brian Akins Lead Systems Engineer CNN Internet Technologies
Re: async write completion prototype
Nicely done. Have you done any benchmarking to see if this improved performance as one would expect? Would it be much more work to use true async IO instead of non blocking IO and polling? What about doing the same for reads, as well as writes? Brian Pane wrote: With the batch of commits I did this weekend, the Event MPM in the async-dev Subversion branch now does write completion in a nonblocking manner. Once an entire response has been generated and passed to the output filter chain, the MPM's poller/listener thread watches the connection for writability events. When the connection becomes writable, the poller thread sends it to one of the worker threads, which writes some more output. At this point, the event-handling code is ready for testing and review by other developers. The main changes on the async-dev branch (compared to the 2.3-dev trunk) are: 1. ap_core_output_filter: rewrite to do nonblocking writes whenever possible. 2. core, http module, and mod_logio: removed the generation of flush buckets where possible. 3. request cleanup and logging: the logger phase and subsequent destruction of the request's pool are now triggered by the destruction of an End-Of-Request bucket in the core output filter. 4. event MPM: asynchronous handling of CONN_STATE_WRITE_COMPLETION. There are several more things that need to be fixed in order to make the asynchronous write completion useful in a production release of httpd-2.x: - The main pollset in the Event MPM currently is sized to hold up to one socket descriptor per worker thread. With asynchronous keepalives and write completion, the pollset should accommodate many descriptors per thread. - The logic for starting more child processes, which Event inherited from Worker, is based on assumptions about the number of concurrent connections being equal to the number of threads. These assumptions aren't valid for a multiple-connections-per-thread MPM. - Similarly, there may be some changes needed in the flow control logic that the listener thread uses to decide whether it can do an accept. - The scoreboard probably needs a redesign. - It may be valuable to have a separate thread pool to run handlers that do arbitrarily lengthy processing, such as mod_perl and mod_php. Brian
Re: [PATCH] Bug 36816: balancer_manager doesn't work if worker name contains port
I agree... For some reason in ap_proxy_get_worker() we specifically skip over the path info, even though it is part of the stored name. This also means that, afaik, we will miss finding valid workers when needed. A forthcoming patch will normalize this.
Re: async write completion prototype
Phillip Susi wrote: Nicely done. Have you done any benchmarking to see if this improved performance as one would expect? Would it be much more work to use true async IO instead of non blocking IO and polling? What about doing the same for reads, as well as writes? All current async_io methods require you to read the data off disk, ie there is no async_sendfile() Reads are much harder in the current httpd. There are several core functions that would need to be rewritten first. -Paul
Re: [pre-release] 2.0.55 *candidate* available for testing
Brian Akins wrote: William A. Rowe, Jr. wrote: Yup - apr 0.9.7 is part of the bundle. We can spell this out in the announce, certainly, and on the downloads page README - would that suffice? How horrible would it be to have the apr_reslist_invalidate patch applied to the bundled apr? Huh? It is already in 0.9.7 :) I committed it to the 0.9.x branch right after 0.9.6 was released. -Paul
Notice of Intent: TR 1.3.34
I will be TR'ing 1.3.34 On Tues or Weds
Re: [pre-release] 2.0.55 *candidate* available for testing
Paul Querna wrote: Huh? It is already in 0.9.7 :) I committed it to the 0.9.x branch right after 0.9.6 was released. Thank you! I guess I didn't check the CHANGELOG closely enough. hangs head in shame / -- Brian Akins Lead Systems Engineer CNN Internet Technologies
Re: async write completion prototype
On what OS? Linux? NT supports async IO on sockets rather nicely, as does FreeBSD iirc. Paul Querna wrote: Phillip Susi wrote: Nicely done. Have you done any benchmarking to see if this improved performance as one would expect? Would it be much more work to use true async IO instead of non blocking IO and polling? What about doing the same for reads, as well as writes? All current async_io methods require you to read the data off disk, ie there is no async_sendfile() Reads are much harder in the current httpd. There are several core functions that would need to be rewritten first. -Paul
Re: [pre-release] 2.0.55 *candidate* available for testing
måndag 10 oktober 2005 16.56 skrev Brian Akins: Paul Querna wrote: Huh? It is already in 0.9.7 :) I committed it to the 0.9.x branch right after 0.9.6 was released. Thank you! I guess I didn't check the CHANGELOG closely enough. i think it's not there :) -- Regards // Oden Eriksson Mandriva: http://www.mandriva.com NUX: http://nux.se
Re: async write completion prototype
Phillip Susi wrote: On what OS? Linux? NT supports async IO on sockets rather nicely, as does FreeBSD iirc. The event MPM doesn't run on NT at all, only Unixes. Yes, FreeBSD (and linux) support async_write().. But this requires that you read the file off of disk, and into a buffer, and then copy it back into the kernel. With a non-blocking sendfile, we can avoid all the data copying (aka 'zero copy'), and let the kernel do everything itself. There is currently no such thing as 'async sendfile', which would be the perfect solution for this use case. There have been various people mentioning it as an idea, but no one has gone out and done it. -Paul
Re: [pre-release] 2.0.55 *candidate* available for testing
Brian Akins wrote: William A. Rowe, Jr. wrote: Yup - apr 0.9.7 is part of the bundle. We can spell this out in the announce, certainly, and on the downloads page README - would that suffice? How horrible would it be to have the apr_reslist_invalidate patch applied to the bundled apr? I'm wondering if we all woke up this morning on different planets :) $ cd httpd-2.0.55 $ grep -r apr_reslist_invalidate * srclib/apr-util/include/apr_reslist.h:APU_DECLARE(apr_status_t) apr_reslist_invalidate(apr_reslist_t *reslist, srclib/apr-util/misc/apr_reslist.c:APU_DECLARE(apr_status_t) apr_reslist_invalidate(apr_reslist_t *reslist, srclib/apr-util/CHANGES: *) Backport the apr_reslist_timeout_set and apr_reslist_invalidate Which patch are you thinking about? The next window of opportunity is apr-0.9.8, which if it's substantial, probably will have its day before 2.0.56. With several other apr-based projects, including arbitrary patches in the release tarball is a non-starter. If you have additional patches for some specific cases, we can consider adding them in dist/httpd/ to patches/apply_to_2.0.55/. Bill
Re: async write completion prototype
On NT you can set the kernel buffer size on the socket to 0 ( with setockopt() or ioctlsocket()? ), and the NIC can DMA directly from the user buffers to send rather than copy to kernel space. This of course, requires that you keep more than one pending async operation so the nic allways has a buffer availible so it can keep the line saturated. If you memory map the source file from the disk, then zero copy IO can be done entirely from user space. Is this optimization not availible on linux or freebsd? As an alternative, you can bypass the cache and do direct async IO to the disk with zero copies. IIRC, this is supported on linux with the O_DIRECT flag. Doing this though, means that you will need to handle caching yourself, which might not be such a good idea. Does Linux not support O_DIRECT on sockets? By using this technique I have been able to achieve TransmitFile() performance levels entirely from user space, without any of the drawbacks of TransmitFile(). Specifically virtually zero cpu time is needed to saturate multiple 100 MBps links, pushing 11,820 KB/s, progress is known the entire time and can be canceled at any time, and a small handfull of threads can service thousonds of clients. Paul Querna wrote: Phillip Susi wrote: On what OS? Linux? NT supports async IO on sockets rather nicely, as does FreeBSD iirc. The event MPM doesn't run on NT at all, only Unixes. Yes, FreeBSD (and linux) support async_write().. But this requires that you read the file off of disk, and into a buffer, and then copy it back into the kernel. With a non-blocking sendfile, we can avoid all the data copying (aka 'zero copy'), and let the kernel do everything itself. There is currently no such thing as 'async sendfile', which would be the perfect solution for this use case. There have been various people mentioning it as an idea, but no one has gone out and done it. -Paul
Re: [pre-release] 2.0.55 *candidate* available for testing
Oden Eriksson wrote: i think it's not there :) Oden, just looked again, would you check your package signature? b45f16a9878e709497820565d42b00b9 httpd-2.0.55.tar.gz and ensure that you are building against the included srclib/apr/ and not against some system installed 0.9.6 version? Bill
Re: async write completion prototype
Phillip Susi wrote: As an alternative, you can bypass the cache and do direct async IO to the disk with zero copies. IIRC, this is supported on linux with the O_DIRECT flag. Doing this though, means that you will need to handle caching yourself, which might not be such a good idea. Does Linux not support O_DIRECT on sockets? Can you not just set the socket to non-blocking using O_NONBLOCK? -- Brian Akins Lead Systems Engineer CNN Internet Technologies
Re: [PATCH] Bug 36816: balancer_manager doesn't work if worker name contains port
For consideration:
Index: modules/proxy/mod_proxy_balancer.c
===
--- modules/proxy/mod_proxy_balancer.c(revision 312628)
+++ modules/proxy/mod_proxy_balancer.c(working copy)
@@ -477,8 +477,12 @@
*val++ = '\0';
if ((tok = ap_strchr(val, '')))
*tok++ = '\0';
+/*
+ * Special case: workers are allowed path information
+ */
if ((access_status = ap_unescape_url(val)) != OK)
-return access_status;
+if (strcmp(args, w) || (access_status !=
HTTP_NOT_FOUND))
+return access_status;
apr_table_setn(params, args, val);
args = tok;
}
@@ -490,13 +494,9 @@
bsel = ap_proxy_get_balancer(r-pool, conf,
apr_pstrcat(r-pool, balancer://, name, NULL));
if ((name = apr_table_get(params, w))) {
-const char *sc = apr_table_get(params, s);
-char *asname = NULL;
-proxy_worker *ws = NULL;
-if (sc) {
-asname = apr_pstrcat(r-pool, sc, ://, name, NULL);
-ws = ap_proxy_get_worker(r-pool, conf, asname);
-}
+proxy_worker *ws;
+
+ws = ap_proxy_get_worker(r-pool, conf, name);
if (ws) {
worker = (proxy_worker *)bsel-workers-elts;
for (n = 0; n bsel-workers-nelts; n++) {
@@ -634,10 +634,10 @@
ap_rvputs(r, tr\ntd, worker-scheme, /
tdtd, NULL);
ap_rvputs(r, a href=\, r-uri, ?b=,
- balancer-name + sizeof(balancer://) - 1,
- s=, worker-scheme, w=, worker-
hostname,
+ balancer-name + sizeof(balancer://) -
1, w=,
+ ap_escape_uri(r-pool, worker-name),
\, NULL);
-ap_rvputs(r, worker-hostname, /a/td, NULL);
+ap_rvputs(r, worker-name, /a/td, NULL);
ap_rvputs(r, td, worker-s-route, NULL);
ap_rvputs(r, /tdtd, worker-s-redirect, NULL);
ap_rprintf(r, /tdtd%d/tdtd, worker-s-
lbfactor);
@@ -678,10 +678,8 @@
ap_rputs( checked, r);
ap_rputs(/tdtr\n, r);
ap_rputs(trtd colspan=2input type=submit value=
\Submit\/td/tr\n, r);
-ap_rvputs(r, /table\ninput type=hidden name=\s\ ,
NULL);
-ap_rvputs(r, value=\, wsel-scheme, \\n, NULL);
-ap_rvputs(r, input type=hidden name=\w\ , NULL);
-ap_rvputs(r, value=\, wsel-hostname, \\n, NULL);
+ap_rvputs(r, /table\ninput type=hidden name=\w\ ,
NULL);
+ap_rvputs(r, value=\, ap_escape_uri(r-pool, wsel-
name), \\n, NULL);
ap_rvputs(r, input type=hidden name=\b\ , NULL);
ap_rvputs(r, value=\, bsel-name + sizeof
(balancer://) - 1,
\\n/form\n, NULL);
Index: modules/proxy/proxy_util.c
===
--- modules/proxy/proxy_util.c(revision 312628)
+++ modules/proxy/proxy_util.c(working copy)
@@ -1218,9 +1218,6 @@
c = strchr(uri, ':');
if (c == NULL || c[1] != '/' || c[2] != '/' || c[3] == '\0')
return NULL;
-/* remove path from uri */
-if ((c = strchr(c + 3, '/')))
-*c = '\0';
worker = (proxy_worker *)conf-workers-elts;
for (i = 0; i conf-workers-nelts; i++) {
Re: [pre-release] 2.0.55 *candidate* available for testing
måndag 10 oktober 2005 17.33 skrev William A. Rowe, Jr.: Oden Eriksson wrote: i think it's not there :) Oden, just looked again, would you check your package signature? b45f16a9878e709497820565d42b00b9 httpd-2.0.55.tar.gz and ensure that you are building against the included srclib/apr/ and not against some system installed 0.9.6 version? Argh! I looked in the wrong place (wrong changelog). Sorry for the noise. Anyway, 2.0.55 works for me on my Mandriva boxes. -- Regards // Oden Eriksson Mandriva: http://www.mandriva.com NUX: http://nux.se
Re: async write completion prototype
Non blocking is not async IO. It is not really possible to perform zero copy IO with non blocking IO semantics, you must have full async IO to issue multiple pending requests. Brian Akins wrote: Phillip Susi wrote: As an alternative, you can bypass the cache and do direct async IO to the disk with zero copies. IIRC, this is supported on linux with the O_DIRECT flag. Doing this though, means that you will need to handle caching yourself, which might not be such a good idea. Does Linux not support O_DIRECT on sockets? Can you not just set the socket to non-blocking using O_NONBLOCK?
Re: [pre-release] 2.0.55 *candidate* available for testing
+1 NetWare Brad On 10/9/2005 at 10:42:43 pm, in message [EMAIL PROTECTED], [EMAIL PROTECTED] wrote: The httpd-2.0.55 candidate, including win32 source .zip and installers*, is now available for testing at http://httpd.apache.org/dev/dist/ Please review this candidate, and when responding, indicate the precise operating system that you have tested. Thank you for your assistance! Bill * note that win32 binary installers were uploaded only at this hour, and it will take up to another two hours for the public server to resync. Thanks for your patience.
Re: [pre-release] 2.0.55 *candidate* available for testing
On Oct 9, 2005, at 9:42 PM, William A. Rowe, Jr. wrote: The httpd-2.0.55 candidate, including win32 source .zip and installers*, As of 17:59 CEST (15 minutes ago), 2.0.55 is running on www.apache.org. Please report any anomalies. We're now also running a very current version of mod_mbox (tagged www.apache.org-20051010). No cores so far but I'll keep an eye on it. S. -- [EMAIL PROTECTED] http://www.temme.net/sander/ PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF smime.p7s Description: S/MIME cryptographic signature
Re: [pre-release] 2.0.55 *candidate* available for testing
måndag 10 oktober 2005 17.28 skrev William A. Rowe, Jr.: Brian Akins wrote: William A. Rowe, Jr. wrote: Yup - apr 0.9.7 is part of the bundle. We can spell this out in the announce, certainly, and on the downloads page README - would that suffice? How horrible would it be to have the apr_reslist_invalidate patch applied to the bundled apr? I'm wondering if we all woke up this morning on different planets :) I must have :) -- Regards // Oden Eriksson Mandriva: http://www.mandriva.com NUX: http://nux.se
Re: [pre-release] 2.0.55 *candidate* available for testing
Sander Temme wrote: On Oct 9, 2005, at 9:42 PM, William A. Rowe, Jr. wrote: The httpd-2.0.55 candidate, including win32 source .zip and installers*, As of 17:59 CEST (15 minutes ago), 2.0.55 is running on www.apache.org. Please report any anomalies. Ack, starting clock with 72 hours to GA, contingent upon an absence of problem reports (specificially regressions). We're now also running a very current version of mod_mbox (tagged www.apache.org-20051010). No cores so far but I'll keep an eye on it. Nice! Hope this licks the earlier chaos. Bill
Re: [pre-release] 2.0.55 *candidate* available for testing
On Mon, Oct 10, 2005 at 10:11:13AM -0600, Brad Nicholes wrote: +1 NetWare Brad BSD/OS using Openssl 0.9.8 is spot on! On 10/9/2005 at 10:42:43 pm, in message [EMAIL PROTECTED], [EMAIL PROTECTED] wrote: The httpd-2.0.55 candidate, including win32 source .zip and installers*, is now available for testing at http://httpd.apache.org/dev/dist/ Please review this candidate, and when responding, indicate the precise operating system that you have tested. Thank you for your assistance! Bill * note that win32 binary installers were uploaded only at this hour, and it will take up to another two hours for the public server to resync. Thanks for your patience. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- Member - Liberal International This is [EMAIL PROTECTED] Ici [EMAIL PROTECTED] God Queen and country! Beware Anti-Christ rising! Better to serve in Heaven that to Rule in Hell. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Re: svn commit: r312704 - in /httpd/httpd/dist: Announcement.html Announcement.txt
[EMAIL PROTECTED] wrote: Author: jim Date: Mon Oct 10 11:05:52 2005 New Revision: 312704 URL: http://svn.apache.org/viewcvs?rev=312704view=rev Log: Reverse preload - prevent site update Jim, nothing is cron'ned up to the website, and this is strictly the /dist/httpd/ location. I think we are safe here letting the translators attack these documents and organize them. The only two folks I expect would push the website up to trunk are you, or me, in the next few days. We can just sync. I'm actually hoping that we blast the announcements in sync, so that 2.0 users aren't compelled to downgrade to 1.3 this time around :) Bill
Re: [pre-release] 2.0.55 *candidate* available for testing
Oden Eriksson wrote: And some investigations told me it requires apr 0.9.7, maybe the autotools stuff should check for this or be documented? Yup - apr 0.9.7 is part of the bundle. We can spell this out in the announce, certainly, and on the downloads page README - would that suffice? There seems to be a bug in the httpd.spec file. It says: Requires: apr = 0.9.5, apr-util 0.9.5 and the devel packages (on the httpd package specs) have no version number: BuildPrereq: apr-devel, apr-util-devel Also, the changelog section in the spec file does not show the upping to 2.0.55. The last documented chenge is for 2.0.53. And it says: changed build to use external apr and apr-util. This confuses me, since apr seems to be present in the 2.0.53 and 2.0.55 tarballs ... Sure enough, 2.0.55 compiles just fine using my old spec file (derived from an old RedHat httpd 2.0.44 spec file). The resulting httpd rpm package does contain libapr and libaprutil 0.9.7. However, with the spec file from the tarball (with the apr BuildPrereq's commented out to get it to confgure/make), I get compile errors. I plead guilty to not having followed the apr-related threads, but I'd expect the RPM spec file to keep me out of trouble with any deps. IMHO, it should rpmbuild right out of the box. If somebody can get me updated on the status of apr (version, bundled or not) and on what the spec file is trying to accomplish (e.g. separate apr packages), I'd probably be able to fix it myself. Luc Pardon Skopos Consulting Belgium
Re: [pre-release] 2.0.55 *candidate* available for testing
William A. Rowe, Jr. wrote: The httpd-2.0.55 candidate, including win32 source .zip and installers*, is now available for testing at http://httpd.apache.org/dev/dist/ My appologies; I should have provided this with the announcement to testers@ and dev@, to avoid confusion; http://httpd.apache.org/dev/dist/CHANGES_2.0.55 includes the summary of the APR libraries as well as HTTP Server, since 2.0.54 (apr 0.9.6). Bill
Re: [pre-release] 2.0.55 *candidate* available for testing
Luc Pardon wrote: Oden Eriksson wrote: And some investigations told me it requires apr 0.9.7, maybe the autotools stuff should check for this or be documented? Yup - apr 0.9.7 is part of the bundle. We can spell this out in the announce, certainly, and on the downloads page README - would that suffice? There seems to be a bug in the httpd.spec file. It says: Requires: apr = 0.9.5, apr-util 0.9.5 and the devel packages (on the httpd package specs) have no version number: BuildPrereq: apr-devel, apr-util-devel Also, the changelog section in the spec file does not show the upping to 2.0.55. The last documented chenge is for 2.0.53. And it says: changed build to use external apr and apr-util. This confuses me, since apr seems to be present in the 2.0.53 and 2.0.55 tarballs ... APR was also present in the 2.0.54 tarball. This was a snafu in the way the rpm change was presented, not in the tarballs. httpd-2.0's distribution tarball will always contain apr 0.9. That doesn't mean httpd-2.2 (with apr 1.x) will do the same; that's yet to be determined. Coming back to rpm's for the moment; I do *not* mean to suggest that this is the best solution for any specific platform or distribution method, be it .rpm, .depot, .pkg, .msi, or any other facility. The problem is that packaging is almost a 20/20 hindsite game. There's no way we should expect that all of these many platform specifics can all be maintained pre-release. That's why, in the Win32 .msi case, there is a seperate httpd/httpd/win32-msi/trunk/ containing the win32 packaging flavor. It doesn't get fixed for a specific release until we know exactly what needed to be fixed :) I'm concerned that the current .spec solution is wrong; it's very platform specific (platform meaning deployment mechanics, in this case, I'm not slamming non-unix rpm implementations). Perhaps we rejigger the tree to httpd/ package/ roll-release/ win32-msi/ rpm/ pkg/ Thoughts? In the interim; is this a showstopper? Do we generally do the right thing (e.g. without changes, can we package up using the existing rpm files?) Obviously 2.0.54 was mispackaged as well, it's minimum apr package dependency should have been 0.9.6 apr, not 0.9.5. Bill
Re: [PATCH] Bug 36816: balancer_manager doesn't work if worker name contains port
On 10/10/2005 05:43 PM, Jim Jagielski wrote:
For consideration:
[..cut..]
Thanks for your thoughts.
BTW: It was a little bit tricky to apply the patch as my Mozilla
seems to have changed things in the mail spaces / empty lines.
So I guess attached patches are easier to handle :-).
I think I found one big problem with the patch:
When we call ap_proxy_get_worker in line 1316 of proxy_util.c
(ap_proxy_pre_request), then the parameter url already contains
the complete URL we must call on the remote server. Example:
ServerName example.com
ProxyPass /mirror/foo http://backend.com/rsync
This would result in a worker named
http://backend.com/rsync
Calling http://example.com/mirror/foo/bar would be translated to
proxy:http://backend.com/rsync/bar in the translate_name hook of mod_proxy.
Thus ap_proxy_get_worker would be called with the URL
http://backend.com/rsync/bar
which would lead to a no match. I guess instead of a simple strcasecmp
we need something like a longest match here since I think we cannot rely
on the order the workers got stored. Do you know if a longest match function
is already at hand somewhere in apr / apr-util or httpd?
BTW: A similar problem should exist with the current code as in this case
ap_proxy_get_worker tries to compare http://backend.com/ with
http://backend.com/rsync
which also fails.
Other smaller problems that need to be solved:
1. Some style and output issues with the manager html interface (e.g.delete
column scheme)
I already did this and extended your patch with these things.
Please have a look if you like the design :-).
2. The xml output is currently broken as it only works with schema and hostname.
I am not quite sure if
1. there is an xml schema already defined for this and needs to be adjusted
2. we need to parse the worker name and replace some of the characters with
xml entities.
I think we need to fix this or if not at least a remark in the documentation
should be made that the xml output is currently broken.
Regards
Rüdiger
Index: modules/proxy/mod_proxy_balancer.c
===
--- modules/proxy/mod_proxy_balancer.c (Revision 312708)
+++ modules/proxy/mod_proxy_balancer.c (Arbeitskopie)
@@ -477,8 +477,12 @@
*val++ = '\0';
if ((tok = ap_strchr(val, '')))
*tok++ = '\0';
+/*
+ * Special case: workers are allowed path information
+ */
if ((access_status = ap_unescape_url(val)) != OK)
-return access_status;
+if (strcmp(args, w) || (access_status !=
HTTP_NOT_FOUND))
+return access_status;
apr_table_setn(params, args, val);
args = tok;
}
@@ -490,13 +494,9 @@
bsel = ap_proxy_get_balancer(r-pool, conf,
apr_pstrcat(r-pool, balancer://, name, NULL));
if ((name = apr_table_get(params, w))) {
-const char *sc = apr_table_get(params, s);
-char *asname = NULL;
-proxy_worker *ws = NULL;
-if (sc) {
-asname = apr_pstrcat(r-pool, sc, ://, name, NULL);
-ws = ap_proxy_get_worker(r-pool, conf, asname);
-}
+proxy_worker *ws;
+
+ws = ap_proxy_get_worker(r-pool, conf, name);
if (ws) {
worker = (proxy_worker *)bsel-workers-elts;
for (n = 0; n bsel-workers-nelts; n++) {
@@ -613,7 +613,7 @@
balancer-name + sizeof(balancer://) - 1,
\, NULL);
ap_rvputs(r, balancer-name, /a/h3\n\n, NULL);
-ap_rputs(\n\ntable border=\0\tr
+ap_rputs(\n\ntable border=\0\ style=\text-align: left;\tr
thStickySession/ththTimeout/ththFailoverAttempts/ththMethod/th
/tr\ntr, r);
ap_rvputs(r, td, balancer-sticky, NULL);
@@ -622,9 +622,9 @@
ap_rprintf(r, td%d/td\n, balancer-max_attempts);
ap_rprintf(r, td%s/td\n,
balancer-lbmethod-name);
-ap_rputs(/table\n, r);
-ap_rputs(\n\ntable border=\0\tr
-thScheme/ththHost/th
+ap_rputs(/table\nbr /, r);
+ap_rputs(\n\ntable border=\0\ style=\text-align: left;\tr
+thWorker URL/th
thRoute/ththRouteRedir/th
thFactor/ththStatus/th
/tr\n, r);
@@ -632,12 +632,11 @@
worker = (proxy_worker *)balancer-workers-elts;
for (n = 0; n balancer-workers-nelts; n++) {
-ap_rvputs(r, tr\ntd, worker-scheme, /tdtd, NULL);
-ap_rvputs(r, a href=\, r-uri, ?b=,
- balancer-name + sizeof(balancer://) - 1,
- s=, worker-scheme, w=, worker-hostname,
+ap_rvputs(r, tr\ntda href=\, r-uri, ?b=,
+
Re: [PATCH] Bug 36816: balancer_manager doesn't work if worker name contains port
On 10/10/2005 11:13 PM, Ruediger Pluem wrote:
[..cut..]
Thus ap_proxy_get_worker would be called with the URL
http://backend.com/rsync/bar
which would lead to a no match. I guess instead of a simple strcasecmp
we need something like a longest match here since I think we cannot rely
on the order the workers got stored. Do you know if a longest match function
Attached a new version which does a longest match and should fix the problem.
[..cut..]
BTW: Tomorrow I will have a look a the log messages, where it makes sense to
exchange
worker-hostname against worker-name.
Regards
Rüdiger
Index: modules/proxy/mod_proxy_balancer.c
===
--- modules/proxy/mod_proxy_balancer.c (Revision 312708)
+++ modules/proxy/mod_proxy_balancer.c (Arbeitskopie)
@@ -477,8 +477,12 @@
*val++ = '\0';
if ((tok = ap_strchr(val, '')))
*tok++ = '\0';
+/*
+ * Special case: workers are allowed path information
+ */
if ((access_status = ap_unescape_url(val)) != OK)
-return access_status;
+if (strcmp(args, w) || (access_status !=
HTTP_NOT_FOUND))
+return access_status;
apr_table_setn(params, args, val);
args = tok;
}
@@ -490,13 +494,9 @@
bsel = ap_proxy_get_balancer(r-pool, conf,
apr_pstrcat(r-pool, balancer://, name, NULL));
if ((name = apr_table_get(params, w))) {
-const char *sc = apr_table_get(params, s);
-char *asname = NULL;
-proxy_worker *ws = NULL;
-if (sc) {
-asname = apr_pstrcat(r-pool, sc, ://, name, NULL);
-ws = ap_proxy_get_worker(r-pool, conf, asname);
-}
+proxy_worker *ws;
+
+ws = ap_proxy_get_worker(r-pool, conf, name);
if (ws) {
worker = (proxy_worker *)bsel-workers-elts;
for (n = 0; n bsel-workers-nelts; n++) {
@@ -613,7 +613,7 @@
balancer-name + sizeof(balancer://) - 1,
\, NULL);
ap_rvputs(r, balancer-name, /a/h3\n\n, NULL);
-ap_rputs(\n\ntable border=\0\tr
+ap_rputs(\n\ntable border=\0\ style=\text-align: left;\tr
thStickySession/ththTimeout/ththFailoverAttempts/ththMethod/th
/tr\ntr, r);
ap_rvputs(r, td, balancer-sticky, NULL);
@@ -622,9 +622,9 @@
ap_rprintf(r, td%d/td\n, balancer-max_attempts);
ap_rprintf(r, td%s/td\n,
balancer-lbmethod-name);
-ap_rputs(/table\n, r);
-ap_rputs(\n\ntable border=\0\tr
-thScheme/ththHost/th
+ap_rputs(/table\nbr /, r);
+ap_rputs(\n\ntable border=\0\ style=\text-align: left;\tr
+thWorker URL/th
thRoute/ththRouteRedir/th
thFactor/ththStatus/th
/tr\n, r);
@@ -632,12 +632,11 @@
worker = (proxy_worker *)balancer-workers-elts;
for (n = 0; n balancer-workers-nelts; n++) {
-ap_rvputs(r, tr\ntd, worker-scheme, /tdtd, NULL);
-ap_rvputs(r, a href=\, r-uri, ?b=,
- balancer-name + sizeof(balancer://) - 1,
- s=, worker-scheme, w=, worker-hostname,
+ap_rvputs(r, tr\ntda href=\, r-uri, ?b=,
+ balancer-name + sizeof(balancer://) - 1, w=,
+ ap_escape_uri(r-pool, worker-name),
\, NULL);
-ap_rvputs(r, worker-hostname, /a/td, NULL);
+ap_rvputs(r, worker-name, /a/td, NULL);
ap_rvputs(r, td, worker-s-route, NULL);
ap_rvputs(r, /tdtd, worker-s-redirect, NULL);
ap_rprintf(r, /tdtd%d/tdtd, worker-s-lbfactor);
@@ -678,10 +677,8 @@
ap_rputs( checked, r);
ap_rputs(/tdtr\n, r);
ap_rputs(trtd colspan=2input type=submit
value=\Submit\/td/tr\n, r);
-ap_rvputs(r, /table\ninput type=hidden name=\s\ , NULL);
-ap_rvputs(r, value=\, wsel-scheme, \\n, NULL);
-ap_rvputs(r, input type=hidden name=\w\ , NULL);
-ap_rvputs(r, value=\, wsel-hostname, \\n, NULL);
+ap_rvputs(r, /table\ninput type=hidden name=\w\ , NULL);
+ap_rvputs(r, value=\, ap_escape_uri(r-pool, wsel-name),
\\n, NULL);
ap_rvputs(r, input type=hidden name=\b\ , NULL);
ap_rvputs(r, value=\, bsel-name + sizeof(balancer://) - 1,
\\n/form\n, NULL);
Index: modules/proxy/proxy_util.c
===
--- modules/proxy/proxy_util.c (Revision 312708)
+++ modules/proxy/proxy_util.c
Re: FileSession automatic cleaning seems to be on vacation
dharana wrote:
I think there is a problem with the autocleaning function in
FileSession. I'm using Mod_Python 3.2.2b in a live server and I today I
noticed this:
/tmp/mp_sess # df -h /tmp
FilesystemSize Used Avail Use% Mounted on
/dev/sda7 989M 544M 395M 58% /tmp
/tmp/mp_sess # df -i /tmp
FilesystemInodes IUsed IFree IUse% Mounted on
/dev/sda7 128768 128768 0 100% /tmp
/tmp/mp_sess # ll /tmp/
total 24
drwxrwxrwt2 root root 4096 Oct 6 11:43 .ICE-unix/
drwx--2 root root16384 Jun 14 22:30 lost+found/
drwxr-x--- 258 httpdsys 4096 Oct 6 06:04 mp_sess/
And, after executing this:
/tmp/mp_sess # find ./ -mmin +1440 -exec rm {} \;
I get this:
/tmp/mp_sess # df -i /tmp
FilesystemInodes IUsed IFree IUse% Mounted on
/dev/sda7 128768 23857 104911 19% /tmp
So I think FileSession isn't doing its work. Any ideas?
Could you check:
ls -l /tmp/mp_sess/.mp_sess.lck
It should only exist when a cleanup is running.
.mp_sess.lck is a lock file to ensure only one cleanup process/thread is
running at one time. If something went wrong during a cleanup
(segfault?) then this lock file might not have been deleted. If an stale
lock file exists any subsequent cleanup will exit without doing
anything. If it exists delete it and see if the cleanup will run to
completion.
If this lock file looks like it might create problems I can add a check
to see if it's stale.
Regards,
Jim
Re: Typo in ./configure --help
dharana wrote: --with-python=DIR Path to specific Python binary It should say: --with-python=PATHPath to specific Python binary --- checking Python version... ./configure: line 2869: /usr/local/hosting/bin/: is a directory ./configure: line 2870: /usr/local/hosting/bin/: is a directory configure: error: This version of mod_python only works with Python major version 2. The one you have seems to be . --- Noted. Will fix. Jim
mod_mbox
The new mod_mbox looks really great. No cores so far, and a big interface improvement. A few comments: 1. Do we really want people subscribing to mailing lists using atom over http? This would consume way more resources than a standard mailing list subscription (due to the polling nature of atom). I don't have any evidence, but this worrys me. 2. There are several formats for each mail message (regular, raw, mime). Probably the links to everything other than the standard format should use the rel=nofollow modifier to keep the search engines out. Keeping the robots off of 2/3 of the links could make a big difference in load considering the number of pages on this site. 3. We should probably turn on the email-address-obfiscation feature. I personally would prefer if everyone could just use proper spam filtering, but I think the general expectation nowadays is that we try to avoid displaying raw addresses. 4. The non-ajax form of the site could probably use a little more attention. It is usuable, but a little dirty in places (overlapping boxes, too-big fonts, etc). Joshua.
Re: mod_mbox
Joshua Slive wrote: 3. We should probably turn on the email-address-obfiscation feature. I personally would prefer if everyone could just use proper spam filtering, but I think the general expectation nowadays is that we try to avoid displaying raw addresses. Consider the other side of that coin, please. Everyone on this list is a regular victim of email impersonation. Let's turn it on for the whole host of reasons. Bill
Re: async write completion prototype
Brian Pane wrote: With the batch of commits I did this weekend, the Event MPM in the async-dev Subversion branch now does write completion in a nonblocking manner. very cool! There are several more things that need to be fixed in order to make the asynchronous write completion useful in a production release of httpd-2.x: ... - The logic for starting more child processes, which Event inherited from Worker, is based on assumptions about the number of concurrent connections being equal to the number of threads. These assumptions aren't valid for a multiple-connections-per-thread MPM. certainly the name MaxClients is wrong - it's really MaxWorkerThreads for event. but the logic does a pretty good job of managing the threads if you can get past the name. - Similarly, there may be some changes needed in the flow control logic that the listener thread uses to decide whether it can do an accept. the flow control I'm aware of is that ap_queue_info_wait_for_idler blocks, therefore the listener temporarily quits accept()ing until worker threads are available. clearly that is needed. the question is should there be some other cap on the number of connections per process. if we do a really good job of raising the connections per thread ratio and continue to use ThreadsPerChild as the throttle, we will be bumping against OS file descriptor per process limits more often. that sounds kinda ugly, so I think we will want some kind of MaxConnectionsPerChild. if the async write completions are moved to the listener thread as Paul suggests, we might want another flow control change. there's no need to reserve/block for a worker thread in that case. - The scoreboard probably needs a redesign. yep. Jeff T and I discussed this offline a while back. a scoreboard slot per connection definately has some appeal. ... my list: - make it work with mod_ssl and http pipelining. this http://mail-archives.apache.org/mod_mbox/httpd-dev/200411.mbox/[EMAIL PROTECTED] fixes it in theory. my problem is testing/verifying it. if I knew how to make mod_ssl's input filters stash data it shouldn't be too bad. - event-ize lingering close. it eats up roughly the same number of worker threads as synchronous writes for SPECweb99. Greg
Re: mod_mbox
On 10/10/05, Joshua Slive [EMAIL PROTECTED] wrote: 1. Do we really want people subscribing to mailing lists using atom over http? This would consume way more resources than a standard mailing list subscription (due to the polling nature of atom). I don't have any evidence, but this worrys me. I guess the question is, what are we concerned about? Is it the bandwidth used by people downloading the feeds? Is it the CPU server side that we need to spend serving them up? Is it both? If it's both, pretty much the only answer is to offload the feed hosting to another service (feedburner or something like that). If it's one or the other there are steps that can be taken. To reduce CPU mod_cache can be used. To reduce bandwidth something like my mod_speedyfeed module can be used to cut down the response bandwidth for clients that send If-Modified-Since and/or Last-Modified headers. You could even combine the two, but keep in mind that mod_speedyfeed burns CPU, and if you use mod_cache alone you end up sending back an entire feed even if a client only cares about a single entry. It just comes down to two questions. Do we want to offer the service, and if so, what resource utilization do we want to optimize for? -garrett
Re: mod_mbox
Garrett Rooney wrote: On 10/10/05, Joshua Slive [EMAIL PROTECTED] wrote: 1. Do we really want people subscribing to mailing lists using atom over http? This would consume way more resources than a standard mailing list subscription (due to the polling nature of atom). I don't have any evidence, but this worrys me. I guess the question is, what are we concerned about? Is it the bandwidth used by people downloading the feeds? Is it the CPU server side that we need to spend serving them up? Is it both? If it's both, pretty much the only answer is to offload the feed hosting to another service (feedburner or something like that). If it's one or the other there are steps that can be taken. To reduce CPU mod_cache can be used. To reduce bandwidth something like my mod_speedyfeed module can be used to cut down the response bandwidth for clients that send If-Modified-Since and/or Last-Modified headers. You could even combine the two, but keep in mind that mod_speedyfeed burns CPU, and if you use mod_cache alone you end up sending back an entire feed even if a client only cares about a single entry. It just comes down to two questions. Do we want to offer the service, and if so, what resource utilization do we want to optimize for? Hell, if we want to optimize for bandwidth, we really should enable mod_deflate, which is pretty damn effective -Paul
Re: [pre-release] 2.0.55 *candidate* available for testing
On Oct 10, 2005, at 7:32 AM, William A. Rowe, Jr. wrote: Brian Pane wrote: I encountered lots of errors in perl-framework's t/TEST with prefork on Darwin 8.2.0/PPC (OS X 10.4.2). I don't yet know whether these are due to httpd-2.0.55 problems or just problems with my Perl installation. Hmmm... review this bug (not fixed in 0.9.7, afaict); http://issues.apache.org/bugzilla/show_bug.cgi?id=34332 It looks like most of the errors I saw were due to environmental problems. I just cleaned up my Perl installation on OS X, and now there's just one test case that's failing now: test 10 of t/apache/limits.t. I _think_ the fix for 34332 is in apr-0.9.7; the CHANGES file includes: *) Fix issue with poll() followed by net I/O yielding EAGAIN on Mac OS 10.4 (Darwin 8). [Wilfredo Sanchez] I probably won't be able to look at that failed test case in limits.t in detail for a few days. If anybody else with OS X has time to test 2.0.55 before then, I'd be grateful. Thanks, Brian
Re: async write completion prototype
On Oct 10, 2005, at 12:01 AM, Paul Querna wrote: Brian Pane wrote: With the batch of commits I did this weekend, the Event MPM in the async-dev Subversion branch now does write completion in a nonblocking manner. Once an entire response has been generated and passed to the output filter chain, the MPM's poller/listener thread watches the connection for writability events. When the connection becomes writable, the poller thread sends it to one of the worker threads, which writes some more output. If the content has already been generated, why add the overhead of the context switch/sending to another thread? Can't the same event thread do a non-blocking write? Once it finishes writing, then yes, we do require a context-switch to another thread to do logging/cleanup. I am mostly thinking about downloading a 1 gig file with the current pattern against a slow client. A non-blocking write might only do ~64k at a time, and causing 1 gig/64k context switches, which seems less than optimal. If I had to choose, I'd rather do the context switches than devote a thread (and the associated stack space) to the connection until the writes are finished--especially if the server is delivering a thousand 1GB files to slow clients concurrently. However, it's probably possible to have _both_ a high ratio of connections to threads (for scalability) and a low ratio of context switches to megabytes delivered (for efficiency). The Event MPM currently has to do a lot of context switching because it detects events in one thread and processes them in another. If we add async write completion to the Leader/Followers MPM (or incorporate a leader/follower thread model into Event), it should reduce the context switches considerably. ... - The main pollset in the Event MPM currently is sized to hold up to one socket descriptor per worker thread. With asynchronous keepalives and write completion, the pollset should accommodate many descriptors per thread. The pollset is auto-resizable. That number is just the maximum number of events that will ever be returned by a single call to _poll(). This number if perfect for the number of threads, since we can never dispatch to more than the number of threads we have... Ah, thanks. I missed that key point. It makes more sense now. Brian
Re: mod_mbox
On Mon, Oct 10, 2005 at 06:46:57PM -0400, Joshua Slive wrote: 1. Do we really want people subscribing to mailing lists using atom over http? This would consume way more resources than a standard mailing list subscription (due to the polling nature of atom). I don't have any evidence, but this worrys me. I thought they were static files? 2. There are several formats for each mail message (regular, raw, mime). Probably the links to everything other than the standard format should use the rel=nofollow modifier to keep the search engines out. Keeping the robots off of 2/3 of the links could make a big difference in load considering the number of pages on this site. *shrug* 3. We should probably turn on the email-address-obfiscation feature. I personally would prefer if everyone could just use proper spam filtering, but I think the general expectation nowadays is that we try to avoid displaying raw addresses. I think this feature is lame (and said so when it was proposed). Spammers are just going to de-obfuscate anyway. Enabling this provides a false layer of security that does no one any good. 4. The non-ajax form of the site could probably use a little more attention. It is usuable, but a little dirty in places (overlapping boxes, too-big fonts, etc). Agreed. -- justin
Re: mod_mbox
Justin Erenkrantz wrote: On Mon, Oct 10, 2005 at 06:46:57PM -0400, Joshua Slive wrote: 1. Do we really want people subscribing to mailing lists using atom over http? This would consume way more resources than a standard mailing list subscription (due to the polling nature of atom). I don't have any evidence, but this worrys me. I thought they were static files? No, it is generated from the mbox files on request. It does include E-Tag and Last-Modified. It could be made into a static file. I proposed doing this originally, but it does add a few issues like creating correct absolute URLs from mod-mbox-util, and no one seemed to really care at the time, so I just made it another handler inside the module, since it was easier. -Paul
nofollow was Re: mod_mbox
2. There are several formats for each mail message (regular, raw, mime). Probably the links to everything other than the standard format should use the rel=nofollow modifier to keep the search engines out. Keeping the robots off of 2/3 of the links could make a big difference in load considering the number of pages on this site. I agree. We don't want Google and friends indexing the raw format, and then ranking it higher than the normal presentation.
Re: mod_mbox
* Justin Erenkrantz [EMAIL PROTECTED] [2005-10-10 22:33:16]: 3. We should probably turn on the email-address-obfiscation feature. I personally would prefer if everyone could just use proper spam filtering, but I think the general expectation nowadays is that we try to avoid displaying raw addresses. I think this feature is lame (and said so when it was proposed). Spammers are just going to de-obfuscate anyway. Enabling this provides a false layer of security that does no one any good. Ok, you are a spam bot and you don't have the slightest idea of who f*** I am. Now, please, de-obfuscate this : [EMAIL PROTECTED] Or even this : [EMAIL PROTECTED] - Sam -- Maxime Petazzoni (http://www.bulix.org) -- gone crazy, back soon. leave message. signature.asc Description: Digital signature
Re: mod_mbox
Maxime Petazzoni wrote: * Justin Erenkrantz [EMAIL PROTECTED] [2005-10-10 22:33:16]: 3. We should probably turn on the email-address-obfiscation feature. I personally would prefer if everyone could just use proper spam filtering, but I think the general expectation nowadays is that we try to avoid displaying raw addresses. I think this feature is lame (and said so when it was proposed). Spammers are just going to de-obfuscate anyway. Enabling this provides a false layer of security that does no one any good. Ok, you are a spam bot and you don't have the slightest idea of who f*** I am. Now, please, de-obfuscate this : [EMAIL PROTECTED] Or even this : [EMAIL PROTECTED] Actually, I feel that last example makes it a pain in the ass for the rest of us humans to email someone about something. -Paul
