mod_proxy Cache-Control: no-cache=directive support Apache 1.3
Hi, Due to a requirement on a project we are currently working on involving using Apache as a caching reverse proxy server to WebLogic. We are considering implementing the Cache-Control: no-cache=directive for the Apache 1.3 mod_proxy module so allow us to prevent the caching of certain headers served by WebLogic. (In particular the session cookie.) Has/is anyone working in this area? Is there any specific reason why this has deliberately not been implemented already? (E.g. performance hit?) Any views on this directive? Thanks, Brian Logica UK Ltd. This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
Re: mod_proxy Cache-Control: no-cache=directive support Apache1.3
On Tue, 19 Feb 2002, Fowler, Brian wrote: Due to a requirement on a project we are currently working on involving using Apache as a caching reverse proxy server to WebLogic. We are considering implementing the Cache-Control: no-cache=directive for the Apache 1.3 mod_proxy module so allow us to prevent the caching of certain headers served by WebLogic. (In particular the session cookie.) I developed mod_accel module. ftp://ftp.lexa.ru/pub/apache-rus/contrib/mod_accel-1.0.13.tar.gz Documentation in Russain only but English translation was started: http://dapi.chaz.ru:8100/articles/mod_accel.xml Features: It allows reverse-proxing only. It frees backend as soon as possible. mod_proxy can keep busy backend with slow client, i.e, using mod_proxy to accelerate backend is not worked with slow clients . It can use busy locks and limit number of connection to backend. It implements primitive fault-tolerance via DNS-balanced backends. It can to cache content with some cookie and ignore another. It can ignore Pragma: no-cache and Authorization. You can specify variuos buffer sizes. It buffers POST body. It logs its state. Drawbacks: I think it can not work in Win32. Probably under Cygwin only. Has/is anyone working in this area? Is there any specific reason why this has deliberately not been implemented already? (E.g. performance hit?) Any views on this directive? mod_proxy is very ancient module and it's hard to maintain it. Igor Ssyeov
Re: other/9871: Server presents wrong certificate with NameVirtualHost
Aaron Bannert wrote: Since the Host: header is part of the encrypted stream, it is not known to the server by the time the cert is required to establish an SSL connection. For this reason it is not possible to do name-based virtual hosting w/ SSL. Perhaps we should make this an explicit failure condition in the mod_ssl code? Name virtual hosting with SSL does work if you have a wildcard certificate - as long as that cert is valid for all the different possible name virtual hosts for the reason you describe above. For this reason I would say leave it as it is. Regards, Graham -- - [EMAIL PROTECTED]There's a moon over Bourbon Street tonight... smime.p7s Description: S/MIME Cryptographic Signature
Re: mod_proxy Cache-Control: no-cache=directive support Apache1.3
Igor Sysoev wrote: mod_proxy is very ancient module and it's hard to maintain it. Er, mod_proxy _was_ a very ancient module, but has been largely overhauled in v1.3 and completely rewritten in v2.0 in both cases having full support of HTTP/1.1. Once mod_cache is finished in v2.0, (in theory) the capability will be there to disengage expensive backends and slow frontends from each other - so all your problems will be solved. :) Regards, Graham -- - [EMAIL PROTECTED]There's a moon over Bourbon Street tonight... smime.p7s Description: S/MIME Cryptographic Signature
Re: mod_proxy Cache-Control: no-cache=directive support Apache1.3
On Tue, 19 Feb 2002, Graham Leggett wrote: Igor Sysoev wrote: mod_proxy is very ancient module and it's hard to maintain it. Er, mod_proxy _was_ a very ancient module, but has been largely overhauled in v1.3 and completely rewritten in v2.0 in both cases having full support of HTTP/1.1. The main problem with mod_proxy is that it reads backend response to 8K buffer and than sends it to client. When it have sent it to client it reads again from backend. After it have sent whole content to client it flushes buffer and only after this it closes backend socket. Even backend send all to its kernel buffer and response is recevied in frontend kernel buffer nevertheless backend need to wait frontend in lingering_close. So we lose at least 2 seconds with small client and big response. Once mod_cache is finished in v2.0, (in theory) the capability will be there to disengage expensive backends and slow frontends from each other - so all your problems will be solved. :) Will see 2.0 but I suppose that multi-threaded mod_perl backend with 10 threads will occupy almost the same memory as 10 mod_perl single thread processes. Igor Sysoev
Re: mod_proxy Cache-Control: no-cache=directive support Apache1.3
Igor Sysoev wrote: The main problem with mod_proxy is that it reads backend response to 8K buffer and than sends it to client. When it have sent it to client it reads again from backend. After it have sent whole content to client it flushes buffer and only after this it closes backend socket. Even backend send all to its kernel buffer and response is recevied in frontend kernel buffer nevertheless backend need to wait frontend in lingering_close. So we lose at least 2 seconds with small client and big response. Will making the 8k buffer bigger solve this problem? I will check that once the end of a request has been detected from the backend, this backend connection is closed before attempting to send the last chunk to the frontend. This should have the effect that with a large enough buffer, the backend will not have to wait around while a slow frontend munches the bytes. Once mod_cache is finished in v2.0, (in theory) the capability will be there to disengage expensive backends and slow frontends from each other - so all your problems will be solved. :) Will see 2.0 but I suppose that multi-threaded mod_perl backend with 10 threads will occupy almost the same memory as 10 mod_perl single thread processes. But a single thread of a mod_perl backend will use less resources if it need only stick around for 100ms, than it will if it has to stick around for a minute. Regards, Graham -- - [EMAIL PROTECTED]There's a moon over Bourbon Street tonight... smime.p7s Description: S/MIME Cryptographic Signature
RE: mod_proxy Cache-Control: no-cache=directive support Apache 1.3
Hi Graham, I don't think mod_proxy does currently conform to RFC2616 in respect of Cache-Control: no-cache=directive It preventing the caching of specific headers, I can't find any code implementing this in the mod_proxy distribution and the following example fails... I have a small test.jsp which just sets Expires, Last-Modified and Cache-Control: no-cache=Set-Cookie headers and configured Apache to act as a Reverse Proxy Server for the page. With an empty cache the initial request gives ... ~ GET -edP http://FCODEVWS2/communities/test.jsp; Cache-Control: no-cache=Set-Cookie Connection: close Date: Tue, 19 Feb 2002 12:37:33 GMT Server: Apache/1.3.23 (Win32) Content-Length: 8 Content-Type: text/html Expires: Tue, 19 Feb 2002 13:06:34 GMT Last-Modified: Tue, 19 Feb 2002 12:06:34 GMT Client-Date: Tue, 19 Feb 2002 12:37:35 GMT Client-Response-Num: 1 Set-Cookie: WebLogicSession=PHJG0oDVfgz68YWMM8DpXAUxFVZ5gugPKyuKQbIf716lkw6bRRBM |-1484200219301769473/-1628821962/6/7001/7001/7002/7002/7001/-1; path=/ X-Cache: MISS from fcomain After which a cached version of the content and headers is created in the cache, but it still contains the Set-Cookie directive which it should not, I don't think ... - 3C72473E 3C723FCA 3C724DDA 0001 3C72470D 3C724710 0008 X-URL: http://fcontdevserver1/communities/test.jsp Connection: close Host: FCODEVWS2 User-Agent: lwp-request/1.40 HTTP/1.1 200 OK Server: Netscape-Enterprise/4.1 Date: Tue, 19 Feb 2002 12:38:22 GMT Content-length: 8 Content-type: text/html Expires: Tue, 19 Feb 2002 13:06:34 GMT Last-modified: Tue, 19 Feb 2002 12:06:34 GMT Cache-control: no-cache=Set-Cookie Set-cookie: WebLogicSession=PHJG0oDVfgz68YWMM8DpXAUxFVZ5gugPKyuKQbIf716lkw6bRRBM|-148420 0219301769473/-1628821962/6/7001/7001/7002/7002/7001/-1; path=/ Test - A subsequent request to the proxy gives a hit to the cache which is correct but it should NOT serve back the Set-Cookie header. ~ GET -edP http://FCODEVWS2/communities/test.jsp; Cache-Control: no-cache=Set-Cookie Connection: close Date: Tue, 19 Feb 2002 12:38:42 GMT Age: 69 Server: Apache/1.3.23 (Win32) Content-Length: 8 Content-Type: text/html Expires: Tue, 19 Feb 2002 13:06:34 GMT Last-Modified: Tue, 19 Feb 2002 12:06:34 GMT Client-Date: Tue, 19 Feb 2002 12:38:40 GMT Client-Response-Num: 1 Set-Cookie: WebLogicSession=PHJG0oDVfgz68YWMM8DpXAUxFVZ5gugPKyuKQbIf716lkw6bRRBM |-1484200219301769473/-1628821962/6/7001/7001/7002/7002/7001/-1; path=/ X-Cache: HIT from fcomain The relevant part of the HTTP spec I think is no-cache If the no-cache directive does not specify a field-name, then a cache MUST NOT use the response to satisfy a subsequent request without successful revalidation with the origin server. This allows an origin server to prevent caching even by caches that have been configured to return stale responses to client requests. If the no-cache directive does specify one or more field-names, then a cache MAY use the response to satisfy a subsequent request, subject to any other restrictions on caching. However, the specified field-name(s) MUST NOT be sent in the response to a subsequent request without successful revalidation with the origin server. This allows an origin server to prevent the re-use of certain header fields in a response, while still allowing caching of the rest of the response. I am missing something? Cheers, Brian -Original Message- From: Graham Leggett [mailto:[EMAIL PROTECTED]] Sent: 19 February 2002 11:59 To: [EMAIL PROTECTED] Subject: Re: mod_proxy Cache-Control: no-cache=directive support Apache 1.3 Fowler, Brian wrote: Due to a requirement on a project we are currently working on involving using Apache as a caching reverse proxy server to WebLogic. We are considering implementing the Cache-Control: no-cache=directive for the Apache 1.3 mod_proxy module so allow us to prevent the caching of certain headers served by WebLogic. (In particular the session cookie.) Has/is anyone working in this area? Is there any specific reason why this has deliberately not been implemented already? (E.g. performance hit?) Any views on this directive? I'm not 100% sure what your question is. The apache proxy follows RFC2616 when it comes to understanding and honouring the Cache-Control directive. The responsibility of setting this header correctly is that of your Weblogic application. If the application is braindead, you should be able to use the capability of mod_headers to add the cache-control header within apache itself. (As I am aware this should work, as long as mod_headers comes before mod_proxy in your config. In v2.0, this should work automatically - but test it first). Regards, Graham -- - [EMAIL PROTECTED]There's a moon over Bourbon Street tonight... This e-mail
Re: building 2.0.32 on Darwin
It appears to me that we are running into a *lot* of problems in (GNU)libtool on OS X and that the libtool guys would appreciate the feedback. Yet, it appears that stuff sent to them never gets included in... has that been other's experience as well? -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: 2.0.32 as GA candidate? Re: [PROPOSAL] 2.0.32 beta announcement
Eli Marmor wrote: Being slashdotted, will be also a good heavy-load test for daedalus, which runs 2.0.32... ;-) Except that, IIRC, daedalis is running prefork exclusively. Has any large site utilized worker? -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: mod_proxy Cache-Control: no-cache=directive support Apache1.3
On Tue, 19 Feb 2002, Graham Leggett wrote: Igor Sysoev wrote: The main problem with mod_proxy is that it reads backend response to 8K buffer and than sends it to client. When it have sent it to client it reads again from backend. After it have sent whole content to client it flushes buffer and only after this it closes backend socket. Even backend send all to its kernel buffer and response is recevied in frontend kernel buffer nevertheless backend need to wait frontend in lingering_close. So we lose at least 2 seconds with small client and big response. Will making the 8k buffer bigger solve this problem? No. It does not resolve 2-second lingering close on backend. I will check that once the end of a request has been detected from the backend, this backend connection is closed before attempting to send the last chunk to the frontend. This should have the effect that with a large enough buffer, the backend will not have to wait around while a slow frontend munches the bytes. 1.3.23 mod_proxy calls ap_proxy_send_fb() and than closes backend. But ap_proxy_send_fb() flushes output to client so it can hang for a long time. Once mod_cache is finished in v2.0, (in theory) the capability will be there to disengage expensive backends and slow frontends from each other - so all your problems will be solved. :) Will see 2.0 but I suppose that multi-threaded mod_perl backend with 10 threads will occupy almost the same memory as 10 mod_perl single thread processes. But a single thread of a mod_perl backend will use less resources if it need only stick around for 100ms, than it will if it has to stick around for a minute. Why it will stick for 100ms only with slow client ? Will Apache 2.0 use separate threads for lingering_close ? Igor Sysoev
Re: building 2.0.32 on Darwin
On Tue, 19 Feb 2002, Pier Fumagalli wrote: Jim Jagielski [EMAIL PROTECTED] wrote: It appears to me that we are running into a *lot* of problems in (GNU)libtool on OS X and that the libtool guys would appreciate the feedback. Yet, it appears that stuff sent to them never gets included in... has that been other's experience as well? Absolutely not. All my patches for libtool (often for esoteric platforms) have been included in a timely manner so far. I sent it several times, never seen a new release coming out with fixes... Gave up :( I have my own patched copy... Apparently those patches were misdirected, because there are no signs of them in the archive of the mailing lists (I grepped through libtool, libtool-patches). If you prefer that, I could resubmit those patches. Just let me know what needs to be submitted.. - Sascha Experience IRCG http://schumann.cx/http://schumann.cx/ircg
Re: building 2.0.32 on Darwin
Sascha Schumann [EMAIL PROTECTED] wrote: On Tue, 19 Feb 2002, Pier Fumagalli wrote: Jim Jagielski [EMAIL PROTECTED] wrote: It appears to me that we are running into a *lot* of problems in (GNU)libtool on OS X and that the libtool guys would appreciate the feedback. Yet, it appears that stuff sent to them never gets included in... has that been other's experience as well? Absolutely not. All my patches for libtool (often for esoteric platforms) have been included in a timely manner so far. I sent it several times, never seen a new release coming out with fixes... Gave up :( I have my own patched copy... Apparently those patches were misdirected, because there are no signs of them in the archive of the mailing lists (I grepped through libtool, libtool-patches). Who knows... I sent them over to [EMAIL PROTECTED] If you prefer that, I could resubmit those patches. Just let me know what needs to be submitted.. Maybe they don't like me :) Attached is my message sent to this list last week about it, with the changes required to compile cleanly 2.0.32 Pier ---BeginMessage--- Pier Fumagalli [EMAIL PROTECTED] wrote: Anyone seen this before? Libtool 1.4.2 doesn't work on OS/X and the modules produced with Apache 2.0 are actually DYLIB(s) and not MH_BUNDLE(s), thus preventing the APR routines to read them... There seems to be a screwup with glibtool's -module parameter... Gotcha... Do a search for xyes and you will see something like archive_cmds=\$nonopt \$(test \\x\$module\\ = xyes echo -bundle... This has to be exchanged into archive_cmds=\$nonopt \$(test \x\$module\ = xyes echo -bundle... To make it work... Any chance we can do something with it? :) Pier ---End Message---
Re: building 2.0.32 on Darwin
Sascha Schumann wrote: On Tue, 19 Feb 2002, Pier Fumagalli wrote: Jim Jagielski [EMAIL PROTECTED] wrote: It appears to me that we are running into a *lot* of problems in (GNU)libtool on OS X and that the libtool guys would appreciate the feedback. Yet, it appears that stuff sent to them never gets included in... has that been other's experience as well? Absolutely not. All my patches for libtool (often for esoteric platforms) have been included in a timely manner so far. They must like you then. :) -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ A society that will trade a little liberty for a little order will lose both and deserve neither - T.Jefferson
Re: other/9871: Server presents wrong certificate with NameVirtualHost
On Tue, Feb 19, 2002 at 01:54:19PM +0200, Graham Leggett wrote: Name virtual hosting with SSL does work if you have a wildcard certificate - as long as that cert is valid for all the different possible name virtual hosts for the reason you describe above. For this reason I would say leave it as it is. Can we detect if a wildcard cert applies to a vhost? If not, perhaps this feature is not explicit enough in the docs? -aaron
Re: Apache 2.0.32 beta is available
On Mon, 18 Feb 2002, Bill Stoddard wrote: On Sun, 17 Feb 2002, Justin Erenkrantz wrote: SNIP SNIP Unless someone submitted patches since the 2.0.32-alpha tarball last week. I don't think Win32/nt works especially as it relies on sockaddr_in for the mpm_winnt. I figured and fixed this, this weekend when I wanted too see if the IPv6 support for Apache on NT worked or not. It did compile but without IPv6 support. After some hours of changing things, enabling getaddrinfo() etc. I suddenly had a IPv6 enabled mpm_winnt version of Apache 2.0.32. I will be cleaning up the diff tonight and fixing the fact that IPv6 addresses get logged quite weirdly, which is a weird offset effect. You'll probably get the diff tomorrow morning CET ;) Cool! Thanks, will be looking for the patch. Bill I haven't made a proper diff, nor the cleanups, but if you copy the files from: http://calvin.ics.hro.nl/~jeroen/apache2/apache_2.0.32-winnt-ipv6.zip And extract them on top of the apache 2.0.32 alpha/beta tarball and compile it should work. This _does_ however require the IPv6 headers supplied by either the platform preview (http://msdn.microsoft.com/downloads/sdks/platform/tpipv6.asp) and or the newer SDK's. Simply build with the InstallBin targets etc and it should all work out fine. The newer IPv6 stacks by MS automatically detect IPv6 presence and use it if possible. This allows for a IPv6 enabled build which should be able to run on most NT's. I say NT's here as the IPv6 stack for 9x/ME is unknown untested for me. Things I have to do to clean it up: - fix the address copying (save_addrinfo goes wrong on me) - put in many #ifdef APR_HAVE_IPV6 lines. - get rid of the debug stuff :) You'll notice that the winnt_mpm now passes the listen_rec to winnt_accept() This was the cleanest way in my opinion for it to get the address family. There is also some weird magic, sockaddr_in6 is normally 24 bytes, but when returned from getaddrinfo() it's always 28 hmm ;) I'll probably have a stab on this tomorrow and pass in a diff to you guys ;) If I am really lucky I will have my adsl line back tomorrow, carrying printed code and using floppydisks isn't fun ;)... I got the sync.. now they have to turn on provisioning... Greets, Jeroen
Re: Request for comments: leader/follower MPM design
Brian Pane wrote: Thanks for catching that. In this particular design, I think we'll be safe because only the current listener thread is ever allowed to pop from the stack. yeah, that occurred to me driving home yesterday. If we can guarantee there is only one popper at a time, that problem can't happen. But if we want a more general apr_atomic_pop which is safe to use with multiple simultaneous pops, we'll have to fix it. But the workaround that you describe below will help solve another problem in the design: I like this technique of CAS'ing two variables at once. In the leader/follower algorithm that I originally posted, there is a race condition that can occur when all the workers are busy and one of them finishes handling its connection at about the same time that the current listener accepts a new connection: worker becoming idle: listener: atomically checks current_listener;waiting for connection finds that there's already a | listener and decides to push | self onto idle_workers stack | [context switch happens here]v |[connection arrives] | | | v |atomically checks the stack; | but the stack is empty, so | it sets current_listener=NULL | | | v | [processing the request] v [thread wakes up here] | v atomically pushes self onto stack In this scenario, we end up with no listener, but there's an idle thread in the stack that ought to have become the new listener. The problem is that the current_listener update and the stack push need to happen atomically. We could solve this problem by combining the current_listener pointer and the top-of-stack pointer into a single 32-bit int so that threads can do a CAS on both at once. Both of these pointers can be replaced by thread IDs, so we can pack them both into a 32-bit value as long as there are fewer than 2^16 worker threads. That should work. I've also seen this type of problem solved with the following: * Check the other guy's stack * If empty, add myself to my stack * Check the other guy's stack again * If it's no longer empty, back out and start over. Both the producer and consumer have to do this. In this case, think of current_listener as being sort of a one deep stack. I hope we will be able to do 64-bit CAS on most 32-bit platforms. That would simplify things greatly. Greg
Re: Apache 2.0.32 beta is available
Jeroen Massar [EMAIL PROTECTED] writes: Unless someone submitted patches since the 2.0.32-alpha tarball last week. I don't think Win32/nt works especially as it relies on sockaddr_in for the mpm_winnt. I figured and fixed this, this weekend when I wanted too see if the IPv6 support for Apache on NT worked or not. It did compile but without IPv6 support. After some hours of changing things, enabling getaddrinfo() etc. I suddenly had a IPv6 enabled mpm_winnt version of Apache 2.0.32. I will be cleaning up the diff tonight and fixing the fact that IPv6 addresses get logged quite weirdly, which is a weird offset effect. You'll probably get the diff tomorrow morning CET ;) I haven't made a proper diff, nor the cleanups, but if you copy the files from: http://calvin.ics.hro.nl/~jeroen/apache2/apache_2.0.32-winnt-ipv6.zip A suggestion: Work on APR first (and submit your patch to [EMAIL PROTECTED]). Get the APR test programs (client, server, sendfile) working nicely. Once that is resolved/mostly resolved, then work on Apache. Some notes on your current code (which I'm guessing at since there is no diff :) ): We would not want to define struct in_addr6 and struct sockaddr_in6 in apr_network_io.h. Please don't use // comments ;) Somebody needs to get to the bottom of the 24-byte/28-byte discrepancy. I guess this is related to you setting salen to sizeof(struct sockaddr_in6)+4 in set_socket_vars(). -- Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site: http://www.geocities.com/SiliconValley/Park/9289/ Born in Roswell... married an alien...
has anybody seen worker segfaults?
For some time some (but after 2.0.32), some tests I run have been segfaulting around the time of a graceful restart. Has anybody else seen something like this? [Tue Feb 19 10:31:43 2002] [notice] child pid 5367 exit signal Segmentation fault (11) [Tue Feb 19 10:31:43 2002] [notice] SIGUSR1 received. Doing graceful restart [Tue Feb 19 10:31:46 2002] [info] mod_unique_id: using ip addr 24.163.40.92 [Tue Feb 19 10:31:47 2002] [notice] Apache/2.0.33-dev (Unix) DAV/2 configured -- resuming normal operations [Tue Feb 19 10:31:47 2002] [info] Server built: Feb 19 2002 10:23:24 [Tue Feb 19 10:31:47 2002] [notice] child pid 5483 exit signal Segmentation fault (11) [Tue Feb 19 10:31:47 2002] [notice] SIGUSR1 received. Doing graceful restart [Tue Feb 19 10:31:50 2002] [info] mod_unique_id: using ip addr 24.163.40.92 [Tue Feb 19 10:31:51 2002] [notice] Apache/2.0.33-dev (Unix) DAV/2 configured -- resuming normal operations [Tue Feb 19 10:31:51 2002] [info] Server built: Feb 19 2002 10:23:24 [Tue Feb 19 10:31:51 2002] [notice] child pid 5579 exit signal Segmentation fault (11) [Tue Feb 19 10:31:51 2002] [notice] SIGUSR1 received. Doing graceful restart The test sends USR1 to the parent then sends in a request (and repeats that cycle 6 or so times). I'm not getting any core dumps from the segfaulting child (threads and Linux :) ). I need to spend more time looking into this, but first I wondered if anybody else saw it. RH 6.2:segfaults as (barely) described above Solaris 8: no segfaults AIX 4.3.3: no segfaults (typical: the platforms where I can get a coredump pretty reliably don't have the problem :) ) Maybe this is a hint... For a couple of the restart iterations, worker on AIX logs this: [crit] ap_queue_push failed with error code -1 Hmmm... -- Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site: http://www.geocities.com/SiliconValley/Park/9289/ Born in Roswell... married an alien...
Re: has anybody seen worker segfaults?
On Tue, Feb 19, 2002 at 12:16:13PM -0500, Jeff Trawick wrote: I'm not getting any core dumps from the segfaulting child (threads and Linux :) ). I need to spend more time looking into this, but first I wondered if anybody else saw it. RH 6.2:segfaults as (barely) described above Solaris 8: no segfaults AIX 4.3.3: no segfaults (typical: the platforms where I can get a coredump pretty reliably don't have the problem :) ) Maybe this is a hint... For a couple of the restart iterations, worker on AIX logs this: [crit] ap_queue_push failed with error code -1 This will only happen in ap_queue_push when apr_thread_mutex_lock or ap_thread_mutex_unlock fail (Yes, I do error checking on the pthread lock/unlock cases *grin*). I'm guessing this is a problem with pthread mutexes on whatever version of linux runs on RH6.2? -aaron
Re: mod_proxy Cache-Control: no-cache=directive support Apache1.3
Igor Sysoev wrote: 1.3.23 mod_proxy calls ap_proxy_send_fb() and than closes backend. But ap_proxy_send_fb() flushes output to client so it can hang for a long time. Then this needs to be fixed. I have 24 hours sitting on a plane starting tomorrow night, so I'll probably have time then :) Why it will stick for 100ms only with slow client ? Will Apache 2.0 use separate threads for lingering_close ? The idea was that instead of passing buckets from the backend to the frontend one-at-a-time, the backend would feed buckets into the memory cache, and independently the frontend would read buckets out of the cache. So: When the backend was finished sending buckets, it went away completely having finished populating the cache. The frontend would then read from the cache taking as long as it liked. There have been a lot of changes to mod_cache though, not sure if this is still in the design. Regards, Graham -- - [EMAIL PROTECTED]There's a moon over Bourbon Street tonight... smime.p7s Description: S/MIME Cryptographic Signature
Re: has anybody seen worker segfaults?
Aaron Bannert [EMAIL PROTECTED] writes: Maybe this is a hint... For a couple of the restart iterations, worker on AIX logs this: [crit] ap_queue_push failed with error code -1 This will only happen in ap_queue_push when apr_thread_mutex_lock or ap_thread_mutex_unlock fail (Yes, I do error checking on the pthread lock/unlock cases *grin*). I'm guessing this is a problem with pthread mutexes on whatever version of linux runs on RH6.2? That log message was seen on AIX, not RH. Since we're guessing, I'll guess that there is a pool lifetime problem (catch-all?) :) To make some progress on this I'll get ap_queue_push() to log something meaningful (which APR call failed, which error) and see if that sheds any light on it (perhaps EFAULT is the error?). -- Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site: http://www.geocities.com/SiliconValley/Park/9289/ Born in Roswell... married an alien...
Re: cvs commit: httpd-2.0/server core.c
[EMAIL PROTECTED] writes: jerenkrantz02/02/18 20:45:53 Modified:.CHANGES include ap_mmn.h http_core.h server core.c Log: Introduce AddOutputFilterByType directive. AddOutputFilterByType DEFLATE text/html Why does this warrant a change to MMN? What module APIs have been changed/removed? (My guess is that it was reasonable to change it anyway to cover API changes like install_transport_filters - pre_connection, but I'm curious about why you changed it with this commit.) -- Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site: http://www.geocities.com/SiliconValley/Park/9289/ Born in Roswell... married an alien...
Re: Request for comments: leader/follower MPM design
Greg Ames wrote: Brian Pane wrote: Thanks for catching that. In this particular design, I think we'll be safe because only the current listener thread is ever allowed to pop from the stack. yeah, that occurred to me driving home yesterday. If we can guarantee there is only one popper at a time, that problem can't happen. But if we want a more general apr_atomic_pop which is safe to use with multiple simultaneous pops, we'll have to fix it. But the workaround that you describe below will help solve another problem in the design: I like this technique of CAS'ing two variables at once. In the leader/follower algorithm that I originally posted, there is a race condition that can occur when all the workers are busy and one of them finishes handling its connection at about the same time that the current listener accepts a new connection: worker becoming idle: listener: atomically checks current_listener;waiting for connection finds that there's already a | listener and decides to push | self onto idle_workers stack | [context switch happens here]v |[connection arrives] | | | v |atomically checks the stack; | but the stack is empty, so | it sets current_listener=NULL | | | v | [processing the request] v [thread wakes up here] | v atomically pushes self onto stack In this scenario, we end up with no listener, but there's an idle thread in the stack that ought to have become the new listener. The problem is that the current_listener update and the stack push need to happen atomically. We could solve this problem by combining the current_listener pointer and the top-of-stack pointer into a single 32-bit int so that threads can do a CAS on both at once. Both of these pointers can be replaced by thread IDs, so we can pack them both into a 32-bit value as long as there are fewer than 2^16 worker threads. That should work. I've also seen this type of problem solved with the following: * Check the other guy's stack * If empty, add myself to my stack * Check the other guy's stack again * If it's no longer empty, back out and start over. Both the producer and consumer have to do this. In this case, think of current_listener as being sort of a one deep stack. I hope we will be able to do 64-bit CAS on most 32-bit platforms. That would simplify things greatly. linux is where the problem lies in general for CAS. the have a cmpxchg function defined for most systems, which does a long (max). I'll get the 32-bit versions uprunning first.. but I don't hold much hope of getting a apr-atomic-cas-64 going for all platforms.. Greg
Re: mod_proxy Cache-Control: no-cache=directive support Apache 1.3
Fowler, Brian wrote: I don't think mod_proxy does currently conform to RFC2616 in respect of Cache-Control: no-cache=directive I just checked - a small note in my copy of RFC2616 says not done yet.. doh...! The relevant part of the HTTP spec I think is no-cache [snip] If the no-cache directive does specify one or more field-names, then a cache MAY use the response to satisfy a subsequent request, subject to any other restrictions on caching. However, the specified field-name(s) MUST NOT be sent in the response to a subsequent request without successful revalidation with the origin server. This allows an origin server to prevent the re-use of certain header fields in a response, while still allowing caching of the rest of the response. So which of the following two senarios is the correct behavior: - If the header specified is not in the request, ignore that Cache-Control header. - If the header IS specified in the request, behave like Cache-Control: max-age=0 OR - If the header specified is not in the request, ignore that Cache-Control header. - If the header IS specified in the request, strip that header and return the request with the header removed. From the text quoted above, either seem valid, or am I reading it wrong...? Regards, Graham -- - [EMAIL PROTECTED]There's a moon over Bourbon Street tonight... smime.p7s Description: S/MIME Cryptographic Signature
xml module docs
If you monitor the CVS list, you have by now noticed that we are converting the apache module docs (ie, the reference manual part of the docs) to xml. I just committed for the first time the xslt transformation into cvs. I did this to assure that we don't need to maintain two sets of module docs in 2.0. Now, to edit module docs, first check if there is both an .xml and .html file. If there is only .html, then just edit the html as usual. If there is an .xml file, then don't edit the .html file, because it is generated from the .xml. Instructions on how to deal with the .xml are here: http://httpd.apache.org/docs-project/docsformat.html Note that, although the final .xml file will need to be transformed to .html, you don't need a full transformation setup to edit the docs and check your works. Recent versions of MSIE, Mozilla, and Netscape will render the xml directly with the xsl stylesheet if you request the .xml in your browser. Joshua.
Re: [PATCH] ap_internal_fast_redirect
Justin Erenkrantz wrote: However, certain modules like mod_deflate attempt to verify that it only deals with non-subreq (!r-main) requests. We can see if we can take that check out - not sure if we can or can not do that. mod_rewrite has r-main checks also. I'd rather not start tinkering with mod_rewrite, at least not until we have a golden 2.x release. Greg
gone sailing
...for about a week and a half, taking a 42' catamaran from Miami to the British Virgin Islands. httpd on daedalus shouldn't need much attention in the mean time. I have a lot of confidence in 2.0.32. Greg
Re: qs values in conjunction with MultiViews
From: Christoph Schneegans [EMAIL PROTECTED] Sent: Tuesday, February 19, 2002 3:56 PM http://httpd.apache.org/docs/content-negotiation.html explains how to use qs parameters in a type map file to assign a source quality to the different variants. It does not explain how to use qs values in conjunction with MultiViews However, it seems to be possible, using the AddType directive: AddType image/svg;qs=0.85 svg svgz AddType image/png;qs=0.7 png AddType image/jpeg;qs=0.8 jpg jpeg This works as expected but I'm concerned about the Content-Type headers that are transmitted to the client: Content-Type: image/svg;qs=0.85 Content-Type: image/png;qs=0.7 Content-Type: image/jpeg;qs=0.8 In contrast to negotiation via type map, the qs values are included in the response header. Good concern, they are invalid. Multiviews was never hacked to decode the qs value [nor were mime or the core.] The patch to do so is not trivial, but is [IMHO] very worthwhile. Bill
Re: gone sailing
On Tue, 19 Feb 2002, Greg Ames wrote: ...for about a week and a half, taking a 42' catamaran from Miami to the British Virgin Islands. httpd on daedalus shouldn't need much attention in the mean time. I have a lot of confidence in 2.0.32. Let me know if you need to have it sailed back :-) Have fun - nice trip in this time of the year. Good current/winds. Dw.
Re: cvs commit: httpd-2.0/server core.c
On Tue, Feb 19, 2002 at 12:52:01PM -0500, Jeff Trawick wrote: [EMAIL PROTECTED] writes: jerenkrantz02/02/18 20:45:53 Modified:.CHANGES include ap_mmn.h http_core.h server core.c Log: Introduce AddOutputFilterByType directive. AddOutputFilterByType DEFLATE text/html Why does this warrant a change to MMN? What module APIs have been changed/removed? (My guess is that it was reasonable to change it anyway to cover API changes like install_transport_filters - pre_connection, but I'm curious about why you changed it with this commit.) A new field was added to core_dir_config. I believe that warrants a bump. If I'm wrong, ooops. -- justin
RE: cvs commit: httpd-2.0/server core.c
On Tue, Feb 19, 2002 at 12:52:01PM -0500, Jeff Trawick wrote: [EMAIL PROTECTED] writes: jerenkrantz02/02/18 20:45:53 Modified:.CHANGES include ap_mmn.h http_core.h server core.c Log: Introduce AddOutputFilterByType directive. AddOutputFilterByType DEFLATE text/html Why does this warrant a change to MMN? What module APIs have been changed/removed? (My guess is that it was reasonable to change it anyway to cover API changes like install_transport_filters - pre_connection, but I'm curious about why you changed it with this commit.) A new field was added to core_dir_config. I believe that warrants a bump. If I'm wrong, ooops. -- justin Core_dir_config should be a private structure, so it doesn't require a MMN bump. Ryan
FW: problem with AddOutputFilterByType directive
On Mon, Feb 18, 2002 at 09:17:19PM -0800, Ian Holsman wrote: it doesn't do what it is supposed to do ALL the time. for example.. take mod-status. inside the handler it decides what type of content-type the program will return. bad module, fix module. That's bogus. Consider mod_cgi, which can't determine the content type until the first bucket is written. I'm with Ian on this one. *) add a filter (with a priority of -1 FTYPE_CONTENT) which just checks the content-type and adds the other filter if it passes. If we have to do this, then I think our architecture is in poor shape. I don't think a filter is the right place to do this - there is way too much overhead in a filter to do this logic. And, we'd also be adding a filter after we've sent data down the filter chain. (Don't get me wrong, I see your rationale behind this, but if the only way to do this is via another filter, we've got problems.) I could be wrong, but I think the way I did it is the right way to do this given what we have now. Anyone else? Am I wrong? -- justin We have a function, ap_pass_brigade(), which is called by every content generator, by definition. Just put a hook into that function. Ryan
Re: qs values in conjunction with MultiViews
William A. Rowe, Jr wrote: [qs values in AddType directives and Content-Type headers] Good concern, they are invalid. Okay, that's a clear statement. Thanks. Multiviews was never hacked to decode the qs value [nor were mime or the core.] But qs values do affect Multiviews! The only task would be to remove them from the response header. Please take a look at http://schneegans.de/temp/qs-multiviews. Using IE or Opera, you'll see the HTML variant. Using Mozilla, you'll see the XHTML variant. The .htaccess file only contains these directives: Options +MultiViews AddType text/html;charset=iso-8859-1;qs=1.000 .html AddType application/xhtml+xml;iso-8859-1;qs=0.999 .xhtml Best regards, Christoph Schneegans
Re: xml module docs
P.S. [please don't flame - just a crazy idea that may improve the procedure of parsing configuration directives] The next step, may be to start (incrementally) to port modules to a new model - where these XML files are used for the first phase of reading directives (mainly their parsing). Of course, the semantics and logics of applying the config directives (after reading and parsing them), will remain the same as today. Since a backword compatibility will be kept temporarily, this step will not be painful. This is like ap_set_..._slot: You can always use the less automatic way, and define your own function. -- Eli Marmor [EMAIL PROTECTED] CTO, Founder Netmask (El-Mar) Internet Technologies Ltd. __ Tel.: +972-9-766-1020 8 Yad-Harutzim St. Fax.: +972-9-766-1314 P.O.B. 7004 Mobile: +972-50-23-7338 Kfar-Saba 44641, Israel
Re: gone sailing
Have a good trip! BTW- digitalroadkill.net will be on 2.0.32 this evening around 10PM CST. Had a lot of issues with timing yesterday, so my apologies. digitalroadkill.net:8080, or 443, if you wish to test. RH 7.2-XFS kernel 2.4.17-xfs-aa glibc-2.2.5 gcc-3.0.3 openssl-0.9.6c PHP 4.1.1 or HTTPD 2.0.32 with SSL support, worker MPM. Drupal(may or may not be installed tonight) AMD 1.333Ghz IDE disk subsystem 256MB PC 133SDRAM On Tue, 2002-02-19 at 16:04, Greg Ames wrote: ...for about a week and a half, taking a 42' catamaran from Miami to the British Virgin Islands. httpd on daedalus shouldn't need much attention in the mean time. I have a lot of confidence in 2.0.32. Greg -- Austin Gonyou Systems Architect, CCNA Coremetrics, Inc. Phone: 512-698-7250 email: [EMAIL PROTECTED] It is the part of a good shepherd to shear his flock, not to skin it. Latin Proverb
Re: FW: problem with AddOutputFilterByType directive
On Tue, Feb 19, 2002 at 03:27:04PM -0800, Ryan Bloom wrote: We have a function, ap_pass_brigade(), which is called by every content generator, by definition. Just put a hook into that function. And have that hook called every time data is sent through a filter (also output filters call ap_pass_brigade)? We can only make this determination once (when we make this decision we have to be sure it is right). But, there also becomes a point that when we positively know the answer, we may be too late to insert arbitrary filters. I understand why you guys are proposing this solution, but I think you're missing my point. Given our current architecture, we have no way of guaranteeing the content-type until most filters have been run. You can't guarantee that the content-type is correct when ap_pass_brigade() is called (first time or many times). You have no idea when the content-type will be set. Any of the filters are free to change the content-type as they execute themselves. Consider the following case: A JSP file that has: % response.setContentType(text/plain) % ...jsp code... At what point do we run this hook that checks content-type? The first time ap_pass_brigade() is called, it may have text/html set. That may be because mod_mime saw .jsp and said, Okay, text/html for .jsp. However, the JSP engine (via an Apache 2.0 filter) says, nah, this should be text/plain. If we base it on the first call to ap_pass_brigade(), we add text/html's filters. If we base it on the second call to ap_pass_brigade, you now need text/plain's filters. Say, we have JSP page that produces PHP code (hey, it's valid in our architecture), and the PHP script now changes it to application/x-ogg. So, the content-type is now something else. It can arbitrarily flip-flop as ap_pass_brigade() is called and the filter tree is walked. However, in our current implementation, once we reach the HTTP_HEADER state, the content-type must be defined. How about we do it as a filter that ran then? Possibly even in ap_http_header_filter. So, let's say we do it right before ap_http_header_filter - we're *guaranteed* to know content_type by then, right? Add the filter as requested by AddOutputFilterByType. Is there a violation of our filter ordering semantics by running this filter out-of-order? We'll be running this filter during HTTP_HEADER. Assume you have two filters - one is explicitly at a higher-priority to ensure that one is always run *after* another. Now, say that the lower-priority filter is added by this directive (but not the other) - we're violating the priorities. I think that's bad. So, perhaps, we could restrict AddOutputFilterByType to only = HTTP_HEADER filters. If it is less than that, we could produce a config-time error. That seems something that might work. Thoughts? I think it may make us too restrictive with this directive though. -- justin
Re: has anybody seen worker segfaults?
On Tue, Feb 19, 2002 at 12:33:58PM -0500, Jeff Trawick wrote: Aaron Bannert [EMAIL PROTECTED] writes: Maybe this is a hint... For a couple of the restart iterations, worker on AIX logs this: [crit] ap_queue_push failed with error code -1 This will only happen in ap_queue_push when apr_thread_mutex_lock or ap_thread_mutex_unlock fail (Yes, I do error checking on the pthread lock/unlock cases *grin*). I'm guessing this is a problem with pthread mutexes on whatever version of linux runs on RH6.2? That log message was seen on AIX, not RH. This would not be the first time pthread problems have been seen under linux though. On RH 7.1 we get segfaults in the middle of the fork call with the prefork mpm under high load. We still haven't been able to figure out why this is happening, but it appears to be a problem with linux pthreads. Has anyone else been having problems with pthreads under linux? -adam -- I believe in Kadath in the cold waste, and Ultima Thule. But you cannot prove to me that Harvard Law School actually exists. - Theodora Goss I'm not like that, I have a cat, I don't need you.. My cat, and about 18 lines of bourne shell code replace you in life. - anonymous Adam Sussman Vidya Media Ventures [EMAIL PROTECTED]
Re: Coredump on daedalus?
Justin Erenkrantz [EMAIL PROTECTED] writes: One so far in /usr/local/apache/corefiles/httpd.core.1, but I'm not sure how this even happened: I looked at this today. It is an oldie but a goodie. We've been getting these very infrequently as long as we've been running 2.0 on daedalus. How did we bypass that rv != APR_SUCCESS check? We should have bailed out since rv == 20132. must be that gdb doesn't know what register rv lives in at the moment And, I think that this tmplen 0 should be tmplen = 0 regardless. Isn't it possible to send 0 bytes? -- justin I've never heard of such a semantic for a send-type call. You should get -1/EAGAIN-or-EWOULDBLOCK. As I suggested to Greg privately, to really nail it as a FreeBSD bug: Right before the sendfile call, assert that we're telling it to send some bytes... right after the sendfile call, assert that if rv == 0 then it sent some bytes The problem is that after putting in such code, it may not hit for 6 months (or 6 minutes), so it is hard to get feedback in a timely basis. -- Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site: http://www.geocities.com/SiliconValley/Park/9289/ Born in Roswell... married an alien...
Re: has anybody seen worker segfaults?
Jeff Trawick wrote:
Maybe this is a hint... For a couple of the restart iterations,
worker on AIX logs this:
[crit] ap_queue_push failed with error code -1
In your AIX test environment, can you catch this error
case in action by putting breakpoints at the two lines
in ap_queue_push() where it's about to return -1?
int ap_queue_push(fd_queue_t *queue, apr_socket_t *sd, apr_pool_t *p,
apr_pool_t **recycled_pool)
{
/*...*/
if (apr_thread_mutex_lock(queue-one_big_mutex) != APR_SUCCESS) {
return FD_QUEUE_FAILURE;
}
/*...*/
if (apr_thread_mutex_unlock(queue-one_big_mutex) != APR_SUCCESS) {
return FD_QUEUE_FAILURE;
}
return FD_QUEUE_SUCCESS;
}
That might help isolate the source of the problem. My two
guesses right now are:
- pool lifetime problem, or
- pthread library problem
--Brian
Re: cvs commit: httpd-2.0/server core.c
On Tue, Feb 19, 2002 at 03:25:23PM -0800, Ryan Bloom wrote: Core_dir_config should be a private structure, so it doesn't require a MMN bump. Just looking at when MMN has been bumped before, it looks like almost all changes to core_dir_config have been followed by MMN bump. Why have people done that? (I'm curious now.) -- justin
