Problem setting new HTTP headers from an input filter
Hi, I have written an Apache module that controls access depending on whether a token is present in the request - normally as an HTTP cookie. I have recently had to modify the module to include an input filter which will spot the token if included in the body of a POST request - the SOAP/xml call to a web service. This works OK. I also need to create some new headers that include user data extracted from the token. The objective is that the new headers are easily picked up as server variables in the application environment. My code uses calls to apr_table_set to create the headers e.g. apr_table_set(r-subprocess_env, HTTP_SAMS_USER, uid); Whilst this code works fine at the _access_checker stage, it seems to have no effect at the input filter processing stage and the extra headers aren't picked up by the cgi application / web service. Is it just not possible to modify the headers at this point in the processing 'pipeline' or is there a different way that I can do it? Many Thanks, Julian Williams
Re: Problem setting new HTTP headers from an input filter
On 10/10/07, Arturo 'Buanzo' Busleiman [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Julian Williams wrote: Is it just not possible to modify the headers at this point in the processing 'pipeline' or is there a different way that I can do it? What TYPE of input filter is it? I'm not sure I know what you mean by TYPE. This is what I have in my register_hooks function: ap_hook_insert_filter(sams_soap_insert, NULL, NULL, APR_HOOK_MIDDLE); ap_register_input_filter(filter_name, sams_soap_filter_in, NULL, AP_FTYPE_RESOURCE ); So perhaps you mean the type AP_FTYPE_RESOURCE . I have also tried using AP_FTYPE_CONTENT_SET but I don't think it made any difference. Thanks, Julian Williams
Re: [PATCH]: mod_cache: don't store headers that will never be used (fwd)
I think that this discussion kind of got lost due to vacations or something... In any case, I'd really like to get some closure. The discussion starts here for those of you that has deleted the thread: http://mail-archives.apache.org/mod_mbox/httpd-dev/200707.mbox/[EMAIL PROTECTED] (the permalink doesn't seem to show the nifty thread list, you have to click a bit for that). What I'd like answered is: - Was the latest patch as suggested OK? - What's the correct way of getting the mod_cache configuration from the mod_disk_cache module? /Nikke -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | [EMAIL PROTECTED] --- Operator...give me the no for 999, QUICK! =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= -- Forwarded message -- From: Niklas Edmundsson [EMAIL PROTECTED] To: dev@httpd.apache.org Date: Wed, 8 Aug 2007 09:28:48 +0200 (MEST) Subject: Re: [PATCH]: mod_cache: don't store headers that will never be used Reply-To: dev@httpd.apache.org X-Bogosity: Unsure, tests=bogofilter, spamicity=0.50, version=0.96.2 On Tue, 31 Jul 2007, Niklas Edmundsson wrote: Any opinions on this? Here's a version with a config directive, defaults to disabled. Silly Q; a directive? Or a env var that can be scoped in interesting ways using mod_setenvif and/or mod_rewrite? Most of our proxy behavior overrides are in terms of envvars. They are much more flexible to being tuned per-browser, per-backend etc. Directive, envvar, I don't think Niklas cares much. Can we make up our mind please? I have no clue on the envvar-stuff, so I don't think I'm qualified to have an opinion. CacheIgnoreCacheControl et al are config directives currently and I have the gut feeling that they should all either be envvar-thingies or config directives, and that starting to mix stuff will only end in confusion and despair ;) I prefer a config-option that I can set serverwide without too much fuss since we want this behaviour on all files. If this can also be accomplished with envvar-stuff then sure. One way might be to do a config directive for now, and deal with the envvar-stuff separately. Related, this config option might also be of interest for mod_disk_cache to enable similar optimizations. What would the good way be to accomplish this? /Nikke /Nikke -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | [EMAIL PROTECTED] --- Now, what was that magic word? Shazam? WHAM! Nah - Garibaldi =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
AW: Filter function is called three times
Hi again, Thanks for your hints. If the request is for a directory, you may see three separate requests, each one ending in the DirectoryIndex options (e.g. /dir/index.html /dir/index.htm, /dir/index.cgi, etc). These are typically subrequests that a single request generates if the request doesn't match a file directly. I tried that out by typing in myurl/filtered/index.html to match the file directly (besides this stuff is configured as Location and not as Directory) with the same results I had when I typed myurl/filtered/. For fully understanding these, try logging the request URI. The request-URI is always /filtered/index.html. Interestingly its even the same if I type in an error. My logs then say that first my input filter is called exactly once. Then the content-handler is called saying File does not exist and then again my filter is called twice more as in the sample before. It could also be related to a browsers ability to request just the headers (via HEAD) or a full document (via GET) for caching. I think I should see that in the access file. Right? In the access file I only see one GET - request. I'd suggest logging a few of the request_rec fields just to be sure. This I haven't tried until now. I'll try that out and see what I can figure out. Thanks for your help again. Sven
Re: AP_CONN_CLOSE on force-response-1.0
On Tuesday 09 October 2007 22:49:38 Jim Jagielski wrote: http://svn.apache.org/viewvc?view=revrevision=78967 That's a 1997 date, btw :) There were no word about broken browsers in that commit, only about broken proxy. ;) On Tuesday 09 October 2007 22:41:19 Jim Jagielski wrote: I can't see changing the behavior now, after all these years. If we want to create a variant that maintains the feasibility of keepalives, then a big +1 for that, but it should be a new envvar, not changing the userland experience of an existing one... And if browser asks explicitly for Keep-Alive, why not to satisfy it? You should keep in mind that implicit behaviour is 'Connection: Close' for 1.0 protocol. I think, new envvar will add unnecessary complexity to configuration and redundant processing to servers...
Re: AP_CONN_CLOSE on force-response-1.0
On Oct 10, 2007, at 6:01 AM, Aleksey Midenkov wrote: On Tuesday 09 October 2007 22:49:38 Jim Jagielski wrote: http://svn.apache.org/viewvc?view=revrevision=78967 That's a 1997 date, btw :) There were no word about broken browsers in that commit, only about broken proxy. ;) On Tuesday 09 October 2007 22:41:19 Jim Jagielski wrote: I can't see changing the behavior now, after all these years. If we want to create a variant that maintains the feasibility of keepalives, then a big +1 for that, but it should be a new envvar, not changing the userland experience of an existing one... And if browser asks explicitly for Keep-Alive, why not to satisfy it? You should keep in mind that implicit behaviour is 'Connection: Close' for 1.0 protocol. I think, new envvar will add unnecessary complexity to configuration and redundant processing to servers... And I think changing the behavior of an existing envvar from how it's been used for ~10years is *sure* to create *more* confusion.
Re: Problem setting new HTTP headers from an input filter
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Julian Williams wrote: Is it just not possible to modify the headers at this point in the processing 'pipeline' or is there a different way that I can do it? What TYPE of input filter is it? - -- Arturo Buanzo Busleiman - Consultor Independiente en Seguridad Informatica Servicios Ofrecidos: http://www.buanzo.com.ar/pro/ Unase a los Foros GNU/Buanzo - La palabra Comunidad en su maxima expresion. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHDMU2AlpOsGhXcE0RCrCnAJ9xWReEtX/Rcp8q0bK0wlMiV30PjQCfU0nt fiYTasO5bdPYrfbrdsBdTh0= =eNW1 -END PGP SIGNATURE-
Re: AP_CONN_CLOSE on force-response-1.0
On Wednesday 10 October 2007 16:25:58 Jim Jagielski wrote: On Oct 10, 2007, at 6:01 AM, Aleksey Midenkov wrote: On Tuesday 09 October 2007 22:49:38 Jim Jagielski wrote: http://svn.apache.org/viewvc?view=revrevision=78967 That's a 1997 date, btw :) There were no word about broken browsers in that commit, only about broken proxy. ;) On Tuesday 09 October 2007 22:41:19 Jim Jagielski wrote: I can't see changing the behavior now, after all these years. If we want to create a variant that maintains the feasibility of keepalives, then a big +1 for that, but it should be a new envvar, not changing the userland experience of an existing one... And if browser asks explicitly for Keep-Alive, why not to satisfy it? You should keep in mind that implicit behaviour is 'Connection: Close' for 1.0 protocol. I think, new envvar will add unnecessary complexity to configuration and redundant processing to servers... And I think changing the behavior of an existing envvar from how it's been used for ~10years is *sure* to create *more* confusion. The behavior is wrong since 2001-03-16 and since then it *sure* made and keeps making confusion. About 6 years.
Re: AP_CONN_CLOSE on force-response-1.0
On Oct 10, 2007, at 8:38 AM, Aleksey Midenkov wrote: On Wednesday 10 October 2007 16:25:58 Jim Jagielski wrote: On Oct 10, 2007, at 6:01 AM, Aleksey Midenkov wrote: On Tuesday 09 October 2007 22:49:38 Jim Jagielski wrote: http://svn.apache.org/viewvc?view=revrevision=78967 That's a 1997 date, btw :) There were no word about broken browsers in that commit, only about broken proxy. ;) On Tuesday 09 October 2007 22:41:19 Jim Jagielski wrote: I can't see changing the behavior now, after all these years. If we want to create a variant that maintains the feasibility of keepalives, then a big +1 for that, but it should be a new envvar, not changing the userland experience of an existing one... And if browser asks explicitly for Keep-Alive, why not to satisfy it? You should keep in mind that implicit behaviour is 'Connection: Close' for 1.0 protocol. I think, new envvar will add unnecessary complexity to configuration and redundant processing to servers... And I think changing the behavior of an existing envvar from how it's been used for ~10years is *sure* to create *more* confusion. The behavior is wrong since 2001-03-16 and since then it *sure* made and keeps making confusion. About 6 years. Whatever. I would for sure wager that if this is changed, people will see a SLEW of incoming reports that Hey, I switched from 2.2.6 to 2.2.7 and I'm seeing this change... I am also sure that wrong or not, there are a lot of people who have either worked around this or are depending on it, and cutting them off at the knees with no workaround is hardly something responsible developers should do. I really don't care all that much, but I tend to recall that we have at least *some* responsibility to our userbase out there, and fixing something to help out one set, while at the same time ignoring the impacts on another set is foolish. BTW: The rev you refer to doesn't change the behavior of force- response-1.0 *itself* but rather when it's applied. So the fact that force- response-1.0 forcibly disables keepalives is something that's been with us for ~10 years.
Re: AW: Filter function is called three times
Sven Kägi wrote: Hi again, Thanks for your hints. If the request is for a directory, you may see three separate requests, each one ending in the DirectoryIndex options (e.g. /dir/index.html /dir/index.htm, /dir/index.cgi, etc). These are typically subrequests that a single request generates if the request doesn't match a file directly. I tried that out by typing in myurl/filtered/index.html to match the file directly (besides this stuff is configured as Location and not as Directory) with the same results I had when I typed myurl/filtered/. For fully understanding these, try logging the request URI. The request-URI is always /filtered/index.html. Interestingly its even the same if I type in an error. My logs then say that first my input filter is called exactly once. Then the content-handler is called saying File does not exist and then again my filter is called twice more as in the sample before. Again, just log the r-uri to see what it is getting called on in your input filter. You may find an /error/404_NOT_FOUND.html reference, which will actually be a sub request due to a missing document. Referencing the file directly should have shown us the file, however, if we are missing an alias, or if we aren't mapping directly to a file that apache knows of, we'll get the 404_NOT_FOUND errors. It could also be related to a browsers ability to request just the headers (via HEAD) or a full document (via GET) for caching. I think I should see that in the access file. Right? In the access file I only see one GET - request. Absolutely. It should show in the access log. I'd suggest logging a few of the request_rec fields just to be sure. This I haven't tried until now. I'll try that out and see what I can figure out. Thanks for your help again. Sven -- Joseph Lewis http://sharktooth.org/ Divide the fire, and you will sooner put it out. - Publius Syrus
Re: AP_CONN_CLOSE on force-response-1.0
On Wednesday 10 October 2007 16:55:03 Jim Jagielski wrote: On Oct 10, 2007, at 8:38 AM, Aleksey Midenkov wrote: The behavior is wrong since 2001-03-16 and since then it *sure* made and keeps making confusion. About 6 years. Whatever. I would for sure wager that if this is changed, people will see a SLEW of incoming reports that Hey, I switched from 2.2.6 to 2.2.7 and I'm seeing this change... I am also sure that wrong or not, there are a lot of people who have either worked around this or are depending on it, and cutting them off at the knees with no workaround is hardly something responsible developers should do. I really don't care all that much, but I tend to recall that we have at least *some* responsibility to our userbase out there, and fixing something to help out one set, while at the same time ignoring the impacts on another set is foolish. Of course it would be foolish. Actually there must be a sensible tradeoff between correctness and backward compatibility. And of course there must be an alternative for those who depend on wrong behaviour. I think in our case the quantity of those who wins from such behaviour is much smaller than of those who loses. In fact, I doubt that there will be numerous complaints if any will be at all. And resolution for those who will suffer can be SetEnvIf Request_Protocol HTTP/1.0 nokeepalive No unnecessary CPU processing for majority. I am sorry if my persistence can appear as a pressure to someone. But I really think this is the best solution. :)
Re: AP_CONN_CLOSE on force-response-1.0
On Wednesday 10 October 2007 18:04:47 Jim Jagielski wrote: On Oct 10, 2007, at 9:38 AM, Aleksey Midenkov wrote: And resolution for those who will suffer can be SetEnvIf Request_Protocol HTTP/1.0 nokeepalive No unnecessary CPU processing for majority. Huh? You're adding another conditional that needs to be checked... And for most cases, where the protocol is 1.1, it would be a waste of time. For one installation from a million it is acceptable tradeoff. But I really doubt it will be necessary to anyone.
Re: AP_CONN_CLOSE on force-response-1.0
On Oct 10, 2007, at 9:38 AM, Aleksey Midenkov wrote: And resolution for those who will suffer can be SetEnvIf Request_Protocol HTTP/1.0 nokeepalive No unnecessary CPU processing for majority. Huh? You're adding another conditional that needs to be checked... And for most cases, where the protocol is 1.1, it would be a waste of time.
Re: AP_CONN_CLOSE on force-response-1.0
On Oct 10, 2007, at 10:08 AM, Aleksey Midenkov wrote: On Wednesday 10 October 2007 18:04:47 Jim Jagielski wrote: On Oct 10, 2007, at 9:38 AM, Aleksey Midenkov wrote: And resolution for those who will suffer can be SetEnvIf Request_Protocol HTTP/1.0 nokeepalive No unnecessary CPU processing for majority. Huh? You're adding another conditional that needs to be checked... And for most cases, where the protocol is 1.1, it would be a waste of time. For one installation from a million it is acceptable tradeoff. But I really doubt it will be necessary to anyone. Or how about leaving the vast majority of the public completely unaffected and creating a new envvar for those who have problems with the 10 year old implementation... If, however, you come up with a complete patch, including docs, that do what you'd like, I'd certainly (assuming it's a valid patch) +1 it.
Re: Broken URI-unescaping in mod_proxy
On Wed, October 10, 2007 3:27 pm, rahul wrote: It would be nice to have different modules for reverse proxy and forward proxy.. from an FTP perspective. There is a fairly large difference in FTP (and perhaps in other protocols too) in terms of the optimizations that needs to be done for forward proxy and reverse proxy. In forward proxy, we can not assume the kind of ftp servers the client requests. So when there is an error of some sort we should try again with a syntax that might be acceptable to the next possible type of server. In the reverse proxy, this is wrong, and introduces unnecessary overheads in network traffic (where it would be simpler to ask the user to provide the type of server in the backend and error out if the ftp server returns error.) There is no need to have separate module to achieve this - providing a mechanism to override certain behaviour when the administrator wants it, but defaulting to RFC compliant behaviour will achieve the same thing. Regards, Graham --
Re: Problem setting new HTTP headers from an input filter
On 10/10/07, Joe Lewis [EMAIL PROTECTED] wrote: Julian Williams wrote: [snip] My code uses calls to apr_table_set to create the headers e.g. apr_table_set(r-subprocess_env, HTTP_SAMS_USER, uid); Try : apr_table_set(r-headers_in, HTTP_SAMS_USER, uid); in addition to the subprocess_env table set? (Just in case you aren't getting to a subprocess?) Unfortunately that doesn't seem to make any difference. My filter also sets the REMOTE_USER header (using r-user) which is being logged correctly in the apache access log. However the web service application doesn't pick it up which suggests that it's environment is setup prior to the filter processing. (It is able to see other standard server variables such as SERVER_NAME) Thanks, Julian Williams
Re: AP_CONN_CLOSE on force-response-1.0
Aleksey Midenkov wrote: The behavior is wrong since 2001-03-16 and since then it *sure* made and keeps making confusion. About 6 years. If so (making confusion), we should see a long history of bugzilla tickets with an impressive CC list on them. Are there? This is not a rhethorical question, the answer might help to setlle the dispute. Regards, Rainer
Re: svn commit: r583466 - in /httpd/httpd/trunk: include/ap_mmn.h modules/proxy/mod_proxy.h modules/proxy/mod_proxy_http.c modules/proxy/proxy_util.c
The whole idea behind this routine is just wrong. That set of characters is insufficient (RFC 3986) and, in any case, a proxy is not responsible for checking valid characters in a URI. Both the original and this new function should be deleted. Roy On Oct 10, 2007, at 6:16 AM, [EMAIL PROTECTED] wrote: Author: jim Date: Wed Oct 10 06:16:56 2007 New Revision: 583466 URL: http://svn.apache.org/viewvc?rev=583466view=rev Log: Abstract out verification of valid encoding via ap_proxy_isvalidenc(). Now we can use it in other proxy protocols. Modified: httpd/httpd/trunk/include/ap_mmn.h httpd/httpd/trunk/modules/proxy/mod_proxy.h httpd/httpd/trunk/modules/proxy/mod_proxy_http.c httpd/httpd/trunk/modules/proxy/proxy_util.c Modified: httpd/httpd/trunk/include/ap_mmn.h URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/include/ ap_mmn.h?rev=583466r1=583465r2=583466view=diff == --- httpd/httpd/trunk/include/ap_mmn.h (original) +++ httpd/httpd/trunk/include/ap_mmn.h Wed Oct 10 06:16:56 2007 @@ -133,6 +133,7 @@ * 20070823.0 (2.3.0-dev) Removed ap_all_available_mutexes_string, * ap_available_mutexes_string for macros * 20070823.1 (2.3.0-dev) add ap_send_interim_response() + * 20070823.2 (2.3.0-dev) add ap_proxy_isvalidenc() * */ Modified: httpd/httpd/trunk/modules/proxy/mod_proxy.h URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/ mod_proxy.h?rev=583466r1=583465r2=583466view=diff == --- httpd/httpd/trunk/modules/proxy/mod_proxy.h (original) +++ httpd/httpd/trunk/modules/proxy/mod_proxy.h Wed Oct 10 06:16:56 2007 @@ -460,6 +460,7 @@ PROXY_DECLARE(request_rec *)ap_proxy_make_fake_req(conn_rec *c, request_rec *r); PROXY_DECLARE(int) ap_proxy_hex2c(const char *x); PROXY_DECLARE(void) ap_proxy_c2hex(int ch, char *x); +PROXY_DECLARE(apr_status_t)ap_proxy_isvalidenc(const char *url, const char *allowed); PROXY_DECLARE(char *)ap_proxy_canonenc(apr_pool_t *p, const char *x, int len, enum enctype t, int forcedec, int proxyreq); PROXY_DECLARE(char *)ap_proxy_canon_netloc(apr_pool_t *p, char **const urlp, char **userp, Modified: httpd/httpd/trunk/modules/proxy/mod_proxy_http.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/ mod_proxy_http.c?rev=583466r1=583465r2=583466view=diff == --- httpd/httpd/trunk/modules/proxy/mod_proxy_http.c (original) +++ httpd/httpd/trunk/modules/proxy/mod_proxy_http.c Wed Oct 10 06:16:56 2007 @@ -37,9 +37,6 @@ const char *err; const char *scheme; apr_port_t port, def_port; -const char *p; -const char *allowed = ~$-_.+!*'(),;:@=/; /* allowed +reserved from - ap_proxy_canonenc */ /* ap_port_of_scheme() */ if (strncasecmp(url, http:, 5) == 0) { @@ -94,15 +91,8 @@ path = ap_proxy_canonenc(r-pool, url, strlen(url), enc_path, 0, r-proxyreq); break; case PROXYREQ_PROXY: -for (p = url; *p; ++p) { -if (!apr_isalnum(*p) !strchr(allowed, *p)) { -if (*p == '%' apr_isxdigit(p[1]) apr_isxdigit (p[2])) { -p += 2; /* an encoded char */ -} -else { -return HTTP_BAD_REQUEST; /* reject bad char in URL */ -} -} +if (ap_proxy_isvalidenc(url, NULL) != APR_SUCCESS) { +return HTTP_BAD_REQUEST; } path = url; break; Modified: httpd/httpd/trunk/modules/proxy/proxy_util.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/ proxy_util.c?rev=583466r1=583465r2=583466view=diff == --- httpd/httpd/trunk/modules/proxy/proxy_util.c (original) +++ httpd/httpd/trunk/modules/proxy/proxy_util.c Wed Oct 10 06:16:56 2007 @@ -135,6 +135,33 @@ } /* + * Confirm that a URL-encoded string only contains + * valid encoding, valid chars are passed in allowed. + * If allowed is NULL, we use useful default. + */ +PROXY_DECLARE(apr_status_t)ap_proxy_isvalidenc(const char *url, + const char *allowed) + +{ +if (!allowed) { +allowed = ~$-_.+!*'(),;:@=/; /* allowed+reserved from + ap_proxy_canonenc */ +} + +for ( ; *url; ++url) { +if (!apr_isalnum(*url) !ap_strchr_c(allowed, *url)) { +if (*url == '%' apr_isxdigit(url[1]) apr_isxdigit (url[2])) { +url += 2; /* an encoded char */ +} +else { +return APR_EGENERAL; /* reject bad char in URL */ +} +} +} +
Re: AP_CONN_CLOSE on force-response-1.0
On Oct 10, 2007, at 7:08 AM, Aleksey Midenkov wrote: On Wednesday 10 October 2007 18:04:47 Jim Jagielski wrote: On Oct 10, 2007, at 9:38 AM, Aleksey Midenkov wrote: And resolution for those who will suffer can be SetEnvIf Request_Protocol HTTP/1.0 nokeepalive No unnecessary CPU processing for majority. Huh? You're adding another conditional that needs to be checked... And for most cases, where the protocol is 1.1, it would be a waste of time. For one installation from a million it is acceptable tradeoff. But I really doubt it will be necessary to anyone. This is all irrelevant. No current installation should need any of those env variables set. They exist solely for working around old versions of old clients that no longer exist on the net. Roy
Re: svn commit: r583466 - in /httpd/httpd/trunk: include/ap_mmn.h modules/proxy/mod_proxy.h modules/proxy/mod_proxy_http.c modules/proxy/proxy_util.c
Good point... forward proxies should just pass through as is, good or bad. Will revert the patch and fix Nick's impl tomorrow when I get back in the office. On Wed, Oct 10, 2007 at 12:39:56PM -0700, Roy T. Fielding wrote: The whole idea behind this routine is just wrong. That set of characters is insufficient (RFC 3986) and, in any case, a proxy is not responsible for checking valid characters in a URI. Both the original and this new function should be deleted. Roy On Oct 10, 2007, at 6:16 AM, [EMAIL PROTECTED] wrote: Author: jim Date: Wed Oct 10 06:16:56 2007 New Revision: 583466 URL: http://svn.apache.org/viewvc?rev=583466view=rev Log: Abstract out verification of valid encoding via ap_proxy_isvalidenc(). Now we can use it in other proxy protocols. Modified: httpd/httpd/trunk/include/ap_mmn.h httpd/httpd/trunk/modules/proxy/mod_proxy.h httpd/httpd/trunk/modules/proxy/mod_proxy_http.c httpd/httpd/trunk/modules/proxy/proxy_util.c Modified: httpd/httpd/trunk/include/ap_mmn.h URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/include/ ap_mmn.h?rev=583466r1=583465r2=583466view=diff == --- httpd/httpd/trunk/include/ap_mmn.h (original) +++ httpd/httpd/trunk/include/ap_mmn.h Wed Oct 10 06:16:56 2007 @@ -133,6 +133,7 @@ * 20070823.0 (2.3.0-dev) Removed ap_all_available_mutexes_string, * ap_available_mutexes_string for macros * 20070823.1 (2.3.0-dev) add ap_send_interim_response() + * 20070823.2 (2.3.0-dev) add ap_proxy_isvalidenc() * */ Modified: httpd/httpd/trunk/modules/proxy/mod_proxy.h URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/ mod_proxy.h?rev=583466r1=583465r2=583466view=diff == --- httpd/httpd/trunk/modules/proxy/mod_proxy.h (original) +++ httpd/httpd/trunk/modules/proxy/mod_proxy.h Wed Oct 10 06:16:56 2007 @@ -460,6 +460,7 @@ PROXY_DECLARE(request_rec *)ap_proxy_make_fake_req(conn_rec *c, request_rec *r); PROXY_DECLARE(int) ap_proxy_hex2c(const char *x); PROXY_DECLARE(void) ap_proxy_c2hex(int ch, char *x); +PROXY_DECLARE(apr_status_t)ap_proxy_isvalidenc(const char *url, const char *allowed); PROXY_DECLARE(char *)ap_proxy_canonenc(apr_pool_t *p, const char *x, int len, enum enctype t, int forcedec, int proxyreq); PROXY_DECLARE(char *)ap_proxy_canon_netloc(apr_pool_t *p, char **const urlp, char **userp, Modified: httpd/httpd/trunk/modules/proxy/mod_proxy_http.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/ mod_proxy_http.c?rev=583466r1=583465r2=583466view=diff == --- httpd/httpd/trunk/modules/proxy/mod_proxy_http.c (original) +++ httpd/httpd/trunk/modules/proxy/mod_proxy_http.c Wed Oct 10 06:16:56 2007 @@ -37,9 +37,6 @@ const char *err; const char *scheme; apr_port_t port, def_port; -const char *p; -const char *allowed = ~$-_.+!*'(),;:@=/; /* allowed +reserved from - ap_proxy_canonenc */ /* ap_port_of_scheme() */ if (strncasecmp(url, http:, 5) == 0) { @@ -94,15 +91,8 @@ path = ap_proxy_canonenc(r-pool, url, strlen(url), enc_path, 0, r-proxyreq); break; case PROXYREQ_PROXY: -for (p = url; *p; ++p) { -if (!apr_isalnum(*p) !strchr(allowed, *p)) { -if (*p == '%' apr_isxdigit(p[1]) apr_isxdigit (p[2])) { -p += 2; /* an encoded char */ -} -else { -return HTTP_BAD_REQUEST; /* reject bad char in URL */ -} -} +if (ap_proxy_isvalidenc(url, NULL) != APR_SUCCESS) { +return HTTP_BAD_REQUEST; } path = url; break; Modified: httpd/httpd/trunk/modules/proxy/proxy_util.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/ proxy_util.c?rev=583466r1=583465r2=583466view=diff == --- httpd/httpd/trunk/modules/proxy/proxy_util.c (original) +++ httpd/httpd/trunk/modules/proxy/proxy_util.c Wed Oct 10 06:16:56 2007 @@ -135,6 +135,33 @@ } /* + * Confirm that a URL-encoded string only contains + * valid encoding, valid chars are passed in allowed. + * If allowed is NULL, we use useful default. + */ +PROXY_DECLARE(apr_status_t)ap_proxy_isvalidenc(const char *url, + const char *allowed) + +{ +if (!allowed) { +allowed = ~$-_.+!*'(),;:@=/; /* allowed+reserved from + ap_proxy_canonenc */ +} + +for ( ; *url;
Re: [PATCH]: mod_cache: don't store headers that will never be used (fwd)
Niklas Edmundsson wrote: What I'd like answered is: - Was the latest patch as suggested OK? The latest patch was the one with a directive, which is +1 from me - though is it possible to add documentation for the directive? - What's the correct way of getting the mod_cache configuration from the mod_disk_cache module? Look inside mod_proxy_http.c for a function called ap_proxy_read_headers(). In it, the module mod_proxy_http reads the config from the module mod_proxy. Regards, Graham --
procattr stuff for 2.2.x
On Fri, Oct 05, 2007 at 09:08:13AM -, [EMAIL PROTECTED] wrote: --- httpd/httpd/branches/2.2.x/STATUS (original) +++ httpd/httpd/branches/2.2.x/STATUS Fri Oct 5 02:08:13 2007 @@ -185,6 +185,10 @@ Not in or needed at trunk/, as apr 1.3.0 has the proper fix. http://people.apache.org/~wrowe/httpd-2.0-2.2-procattr-bugfix-log.c.patch +1: wrowe + rpluem says: Is this really the correct thing to do on UNIX? I am not sure + if all dup2 implementation notice that both fd's are the same. Otherwise + they close stdout/stderr first and dup a then closed fd in stdout/stderr, + leaving us without stdout/stderr in the child. Before Bill loses the will to hack the piped logging code... is this all correct: 1) r452431 introduced a regression on Win32 2) PR 40651 (fixed by r452431) was Unix-specific? Or at least PR 40651 is not as harmful as the aforementioned regression. 3) we can restore the pre-2.2.4 behaviour on Win32 by doing something simple like the below, without having to mess with the APR procattr code or continue adding complexity to the log.c code? Index: server/log.c === --- server/log.c(revision 583629) +++ server/log.c(working copy) @@ -265,6 +265,11 @@ apr_proc_t *procnew; apr_file_t *errfile; +#ifdef WIN32 +/* workaround for APR 1.2.x */ +dummy_stderr = 0; +#endif + if (((rc = apr_procattr_create(procattr, p)) == APR_SUCCESS) ((rc = apr_procattr_cmdtype_set(procattr, APR_SHELLCMD_ENV)) == APR_SUCCESS)
mod_ssl and OCSP stapling
OSSI has been working with the OpenSSL team (Dr. Steve Henson) to support OCSP stapling (RFC 3546, section 3.6 Certificate Status Request) for the Mozilla foundation. To date OCSP stapling has been implemented in the OpenSSL dev branch and will appear in the upcoming 0.9.8g release. We're now looking at OCSP support for mod_ssl, with the goal of producing a working patch, and will be floating some queries as we go. We would like to see this patch, or some comparable implementation, incorporated into mod_ssl. Steve is confident he can put together something that works, but there are several design choices to be made on the way that we would like input on. For instance: - A tie-in to mod_proxy is intriguing but perhaps too ambitious at present. How desirable would that be? - What about integrating the OCSP stapling with the somewhat complementary Marc Stern OCSP patch? There will be some overlap with OCSP responder query code. If anyone has insights to offer or would like to collaborate please sing out. There should be a patch in about a week for bug tracker comments. Thanks, -Steve M. -- Steve Marquess Open Source Software institute [EMAIL PROTECTED]
[STATUS] (httpd-trunk) Wed Oct 10 23:49:52 2007
APACHE 2.3 STATUS: -*-text-*- Last modified at [$Date: 2006-08-22 16:41:03 -0400 (Tue, 22 Aug 2006) $] The current version of this file can be found at: * http://svn.apache.org/repos/asf/httpd/httpd/trunk/STATUS Documentation status is maintained seperately and can be found at: * docs/STATUS in this source tree, or * http://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/STATUS Consult the following STATUS files for information on related projects: * http://svn.apache.org/repos/asf/apr/apr/trunk/STATUS * http://svn.apache.org/repos/asf/apr/apr-util/trunk/STATUS Patches considered for backport are noted in their branches' STATUS: * http://svn.apache.org/repos/asf/httpd/httpd/branches/1.3.x/STATUS * http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x/STATUS * http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x/STATUS Release history: [NOTE that x.{odd}.z versions are strictly Alpha/Beta releases, while x.{even}.z versions are Stable/GA releases.] 2.3.0 : in development Contributors looking for a mission: * Just do an egrep on TODO or XXX in the source. * Review the bug database at: http://issues.apache.org/bugzilla/ * Review the PatchAvailable bugs in the bug database: https://issues.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDproduct=Apache+httpd-2keywords=PatchAvailable After testing, you can append a comment saying Reviewed and tested. * Open bugs in the bug database. CURRENT RELEASE NOTES: RELEASE SHOWSTOPPERS: * Handling of non-trailing / config by non-default handler is broken http://marc.theaimsgroup.com/?l=apache-httpd-devm=105451701628081w=2 jerenkrantz asks: Why should this block a release? wsanchez agrees: this may be a change in behavior, but isn't clearly wrong, and even if so, it doesn't seem like a showstopper. * the edge connection filter cannot be removed http://marc.theaimsgroup.com/?l=apache-httpd-devm=105366252619530w=2 jerenkrantz asks: Why should this block a release? stas replies: because it requires a rewrite of the filters stack implementation (you have suggested that) and once 2.2 is released you can't do that anymore. CURRENT VOTES: * If the parent process dies, should the remaining child processes gracefully self-terminate. Or maybe we should make it a runtime option, or have a concept of 2 parent processes (one being a hot spare). See: Message-ID: [EMAIL PROTECTED] Self-destruct: Ken, Martin, Lars Not self-destruct: BrianP, Ian, Cliff, BillS Make it runtime configurable: Aaron, jim, Justin, wrowe, rederpj, nd /* The below was a concept on *how* to handle the problem */ Have 2 parents: +1: jim -1: Justin, wrowe, rederpj, nd +0: Lars, Martin (while standing by, could it do something useful?) * Make the worker MPM the default MPM for threaded Unix boxes. +1: Justin, Ian, Cliff, BillS, striker, wrowe, nd +0: BrianP, Aaron (mutex contention is looking better with the latest code, let's continue tuning and testing), rederpj, jim -0: Lars pquerna: Do we want to change this for 2.2? RELEASE NON-SHOWSTOPPERS BUT WOULD BE REAL NICE TO WRAP THESE UP: * Patches submitted to the bug database: http://issues.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDproduct=Apache+httpd-2keywords=PatchAvailable * Filter stacks and subrequests, redirects and fast redirects. There's at least one PR that suffers from the current unclean behaviour (which lets the server send garbage): PR 17629 nd says: Every subrequest should get its own filter stack with the subreq_core filter as bottom-most. That filter does two things: - swallow EOS buckets - redirect the data stream to the upper request's (rr-main) filter chain directly after the subrequest's starting point. Once we have a clean solution, we can try to optimize it, so that the server won't be slow down too much. * RFC 2616 violations. Closed PRs: 15857. Open PRs: 15852, 15859, 15861, 15864, 15865, 15866, 15868, 15869, 15870, 16120, 16125, 16126, 16133, 16135, 16136, 16137, 16138, 16139, 16140, 16142, 16518, 16520, 16521, jerenkrantz says: need to decide how many we need to backport and/or if these rise to showstopper status. wrowe suggests: it would be nice to see MUST v.s. SHOULD v.s. MAY out of this list, without reviewing them individually. * There is a bug in how we sort some hooks, at
[STATUS] (httpd-2.0) Wed Oct 10 23:47:01 2007
APACHE 2.0 STATUS: -*-text-*- Last modified at [$Date: 2007-10-10 14:13:31 -0400 (Wed, 10 Oct 2007) $] The current version of this file can be found at: * http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x/STATUS Documentation status is maintained seperately and can be found at: * docs/STATUS in this source tree, or * http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x/docs/STATUS Consult the following STATUS files for information on related projects: * http://svn.apache.org/repos/asf/apr/apr/branches/0.9.x/STATUS * http://svn.apache.org/repos/asf/apr/apr-util/branches/0.9.x/STATUS Consult the trunk/ for all new development and documentation efforts: * http://svn.apache.org/repos/asf/httpd/httpd/trunk/STATUS * http://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/STATUS Release history: 2.0.62 : In maintenance 2.0.61 : Released September 7, 2007. 2.0.60 : Tagged August 10, 2007, not released. 2.0.59 : released July 28, 2006 as GA. 2.0.58 : released May 1, 2006 as GA. 2.0.57 : tagged April 19, 2006, not released. 2.0.56 : tagged April 16, 2006, not released. 2.0.55 : released October 16, 2005 as GA. 2.0.54 : released April 17, 2005 as GA. 2.0.53 : released February 7, 2005 as GA. 2.0.52 : released September 28, 2004 as GA. 2.0.51 : released September 15, 2004 as GA. 2.0.50 : released June 30, 2004 as GA. 2.0.49 : released March 19, 2004 as GA. 2.0.48 : released October 29, 2003 as GA. 2.0.47 : released July 09, 2003 as GA. 2.0.46 : released May 28, 2003 as GA. 2.0.45 : released April 1, 2003 as GA. 2.0.44 : released January 20, 2003 as GA. 2.0.43 : released October 3, 2002 as GA. 2.0.42 : released September 24, 2002 as GA. 2.0.41 : rolled September 16, 2002. not released. 2.0.40 : released August 9, 2002 as GA. 2.0.39 : released June 17, 2002 as GA. 2.0.38 : rolled June 16, 2002. not released. 2.0.37 : rolled June 11, 2002. not released. 2.0.36 : released May 6, 2002 as GA. 2.0.35 : released April 5, 2002 as GA. 2.0.34 : tagged March 26, 2002. 2.0.33 : tagged March 6, 2002. not released. 2.0.32 : released Feburary 16, 2002 as beta. 2.0.31 : rolled Feburary 1, 2002. not released. 2.0.30 : tagged January 8, 2002. not rolled. 2.0.29 : tagged November 27, 2001. not rolled. 2.0.28 : released November 13, 2001 as beta. 2.0.27 : rolled November 6, 2001 2.0.26 : tagged October 16, 2001. not rolled. 2.0.25 : rolled August 29, 2001 2.0.24 : rolled August 18, 2001 2.0.23 : rolled August 9, 2001 2.0.22 : rolled July 29, 2001 2.0.21 : rolled July 20, 2001 2.0.20 : rolled July 8, 2001 2.0.19 : rolled June 27, 2001 2.0.18 : rolled May 18, 2001 2.0.17 : rolled April 17, 2001 2.0.16 : rolled April 4, 2001 2.0.15 : rolled March 21, 2001 2.0.14 : rolled March 7, 2001 2.0a9 : released December 12, 2000 2.0a8 : released November 20, 2000 2.0a7 : released October 8, 2000 2.0a6 : released August 18, 2000 2.0a5 : released August 4, 2000 2.0a4 : released June 7, 2000 2.0a3 : released April 28, 2000 2.0a2 : released March 31, 2000 2.0a1 : released March 10, 2000 Contributors looking for a mission: * Just do an egrep on TODO or XXX in the source. * Review the bug database at: http://issues.apache.org/bugzilla/ * Review the PatchAvailable bugs in the bug database: http://issues.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDproduct=Apache+httpd-2.0keywords=PatchAvailable After testing, you can append a comment saying Reviewed and tested. * Open bugs in the bug database. CURRENT RELEASE NOTES: * Forward binary compatibility is expected of Apache 2.0.x releases, such that no MMN major number changes will occur. Such changes can only be made in the trunk. * All commits to branches/2.0.x must be reflected in SVN trunk, as well, if they apply. Logical progression is commit to trunk, get feedback and votes on list or in STATUS, then merge into branches/2.2.x, and finally merge into branches/2.0.x, as applicable. RELEASE SHOWSTOPPERS: * core log.c: Authored and Reviewed by both rplume and wrowe within the same 10 minutes, share only a single apr_file_t/fd between the stderr and server_main-error_log to prevent any lingering write handles from hanging around in unexpected ways. http://svn.apache.org/viewvc?view=revrevision=580437 PR 43491, solution validated by reporter +1: wrowe, rpluem * core log.c: Work around possible solutions rejected by apr for the old implementation of apr_proc_create(), and explicitly pass the output and error channels to all log processes created.
[STATUS] (httpd-2.2) Wed Oct 10 23:47:17 2007
APACHE 2.2 STATUS: -*-text-*- Last modified at [$Date: 2007-10-09 20:28:28 -0400 (Tue, 09 Oct 2007) $] The current version of this file can be found at: * http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x/STATUS Documentation status is maintained seperately and can be found at: * docs/STATUS in this source tree, or * http://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/STATUS Consult the following STATUS files for information on related projects: * http://svn.apache.org/repos/asf/apr/apr/trunk/STATUS * http://svn.apache.org/repos/asf/apr/apr-util/trunk/STATUS Patches considered for backport are noted in their branches' STATUS: * http://svn.apache.org/repos/asf/httpd/httpd/branches/1.3.x/STATUS * http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x/STATUS * http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x/STATUS Release history: [NOTE that x.{odd}.z versions are strictly Alpha/Beta releases, while x.{even}.z versions are Stable/GA releases.] 2.2.7 : In development 2.2.6 : Released September 7, 2007. 2.2.5 : Tagged August 10, 2007, not released. 2.2.4 : Released on January 9, 2007 as GA. 2.2.3 : Released on July 28, 2006 as GA. 2.2.2 : Released on May 1, 2006 as GA. 2.2.1 : Tagged on April 1, 2006, not released. 2.2.0 : Released on December 1, 2005 as GA. 2.1.10 : Tagged on November 19, 2005, not released. 2.1.9 : Released on November 5, 2005 as beta. 2.1.8 : Released on October 1, 2005 as beta. 2.1.7 : Released on September 12, 2005 as beta. 2.1.6 : Released on June 27, 2005 as alpha. 2.1.5 : Tagged on June 17, 2005. 2.1.4 : not released. 2.1.3 : Released on February 22, 2005 as alpha. 2.1.2 : Released on December 8, 2004 as alpha. 2.1.1 : Released on November 19, 2004 as alpha. 2.1.0 : not released. Contributors looking for a mission: * Just do an egrep on TODO or XXX in the source. * Review the bug database at: http://issues.apache.org/bugzilla/ * Review the PatchAvailable bugs in the bug database: https://issues.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDproduct=Apache+httpd-2keywords=PatchAvailable After testing, you can append a comment saying Reviewed and tested. * Open bugs in the bug database. CURRENT RELEASE NOTES: * Forward binary compatibility is expected of Apache 2.2.x releases, such that no MMN major number changes will occur. Such changes can only be made in the trunk. * All commits to branches/2.2.x must be reflected in SVN trunk, as well, if they apply. Logical progression is commit to trunk, get feedback and votes on list or in STATUS, then merge into branches/2.2.x, as applicable. RELEASE SHOWSTOPPERS: PATCHES ACCEPTED TO BACKPORT FROM TRUNK: [ start all new proposals below, under PATCHES PROPOSED. ] PATCHES PROPOSED TO BACKPORT FROM TRUNK: [ New proposals should be added at the end of the list ] * mpm_winnt: Eliminate wait_for_many_objects. Allows the clean shutdown of the server when the MaxClients is higher then 257, in a more responsive manner. Trunk version of patch: http://svn.apache.org/viewvc?view=revrevision=573103 http://svn.apache.org/viewvc?view=revrevision=573105 2.2.x version of patch: http://people.apache.org/~wrowe/mpm_winnt_shutdown-2.2.patch +1: wrowe * mod_authn_dbd: Export any additional columns queried in the SQL select into the environment with the name AUTHENTICATE_COLUMN. This brings mod_authn_dbd behaviour in line with mod_authnz_ldap. Trunk: http://svn.apache.org/viewvc?view=revrevision=466865 http://svn.apache.org/viewvc?view=revrevision=571798 http://svn.apache.org/viewvc?view=revrevision=571804 +1: minfrin rpluem says: r466865 has a conflict in modules/aaa/mod_auth.h r571804 has a conflict in docs/manual/mod/mod_authnz_ldap.xml Without r571838 the documentation for mod_authn_dbd fails to build. * multiple files, Trivial cleanups PR: 39518 - Christophe JAILLET http://svn.apache.org/viewvc?view=revrevision=557837 http://svn.apache.org/viewvc?view=revrevision=557972 +1: rpluem niq: this isn't a straight backport (which is why I dropped it). +1 for core, modules/dav, modules/filters, and modules/ssl Not Applicable to modules/aaa * mod_include: Add an if directive syntax to test whether an URL is accessible, and if so, conditionally display content. This allows a webmaster to hide a link to a private page when the user has no access to that page. http://svn.apache.org/viewvc?view=revrevision=571872 http://svn.apache.org/viewvc?view=revrevision=571927
Re: procattr stuff for 2.2.x
Joe Orton wrote: 3) we can restore the pre-2.2.4 behaviour on Win32 by doing something simple like the below, without having to mess with the APR procattr code or continue adding complexity to the log.c code? -1 not acceptable, you are looping back to dangling pipes, lingering processes, non-running code, etc. 2.2.4 and prior code didn't even run (piped loggers) on win32, remember? Unix had it's own issues. The 2.2.4-and-prior code was not acceptable, the proposed hack is similarly unacceptable. It was flawed on so many levels, the proposed patch is as close as I can come to a comprise given the broken state of APR (due entirely to your myopic interpretation of the versioning policy, which I'm through arguing.) Understand that is a /proposal/, it's still going to take me a few days to resolve a test/aprpipe.c fault on win32 (one of three on win32, of which the other two are well-known waiting on apr 2.0). Actually about 8 last hours of work to fix that and vet this patch as thoroughly as the first rounds, but I'm distracted thurs/fri and I'll see how much attention I can apply to it before the weekend. I appreciate your desire to avoid polluting the unix path, but it's really senseless to optimize code that's only invoked once/ process, and incredibly destructive to keep altering the code paths as we saw over the past few months. Keep a single impl. We have it right in 1.3.0 and I've already flipped the 'apr 1.3 required' bits in httpd trunk. Remember this audit already had uncloaked a host of UNIX issues. One code path. If the proposed patch leaves dangling handles, it's because this is a draft that I haven't finished vetting, I just wanted to get it out there so folks would have a preview. The final code will be clean, a few redundant dups in a rarely executed code block, but clean of dangling pipes on any platform. --- server/log.c(revision 583629) +++ server/log.c(working copy) @@ -265,6 +265,11 @@ apr_proc_t *procnew; apr_file_t *errfile; +#ifdef WIN32 +/* workaround for APR 1.2.x */ +dummy_stderr = 0; +#endif + if (((rc = apr_procattr_create(procattr, p)) == APR_SUCCESS) ((rc = apr_procattr_cmdtype_set(procattr, APR_SHELLCMD_ENV)) == APR_SUCCESS)