Header with trailing space in field name passed to CGI script
Hello, According to the rfc822#section-3.2, SPACE character is not allowed in the header field name. I have tested apache httpd version 2.2.16 and 2.4.12 and with both installations a header with the trailing space in the field name is passed to a CGI script's environment (in both cases mod_cgid is active if that matters). Attached you can find two simple tests and a simple CGI script used for these tests: in first case used one header: X-MSISDNSPACE:SPACEtest it is passed to CGI script as HTTP_X_MSISDN variable. in the second case I used two headers: X-MSISDNSPACE:SPACEtest X-MSISDN:SPACEtest2 both translate to the same HTTP_X_MSISDN environment variable with two values separated by the comma. Is there any reason to not ignore a header with the trailing space in the field name and pass it to the CGI environment? If not, is it a bug or I am missing something? Thanks in advance, -- George Chelidze $ telnet localhost 80 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GET /cgi-bin/test.pl HTTP/1.0 Host: localhost X-MSISDN : test HTTP/1.1 200 OK Date: Tue, 14 Apr 2015 05:48:52 GMT Server: Apache/2.4.12 (Unix) Connection: close Content-Type: text/html CONTEXT_DOCUMENT_ROOT = /opt/apache-httpd-2.4.12/cgi-bin/ CONTEXT_PREFIX = /cgi-bin/ DOCUMENT_ROOT = /opt/apache-httpd-2.4.12/htdocs GATEWAY_INTERFACE = CGI/1.1 HTTP_HOST = localhost HTTP_X_MSISDN = test LD_LIBRARY_PATH = /opt/apache-httpd-2.4.12/lib PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QUERY_STRING = REMOTE_ADDR = 127.0.0.1 REMOTE_PORT = 48017 REQUEST_METHOD = GET REQUEST_SCHEME = http REQUEST_URI = /cgi-bin/test.pl SCRIPT_FILENAME = /opt/apache-httpd-2.4.12/cgi-bin/test.pl SCRIPT_NAME = /cgi-bin/test.pl SERVER_ADDR = 127.0.0.1 SERVER_ADMIN = y...@example.com SERVER_NAME = localhost SERVER_PORT = 80 SERVER_PROTOCOL = HTTP/1.0 SERVER_SIGNATURE = SERVER_SOFTWARE = Apache/2.4.12 (Unix) Connection closed by foreign host. $ telnet localhost 80 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GET /cgi-bin/test.pl HTTP/1.0 Host: localhost X-MSISDN : test X-MSISDN: test2 HTTP/1.1 200 OK Date: Tue, 14 Apr 2015 05:54:45 GMT Server: Apache/2.4.12 (Unix) Connection: close Content-Type: text/html CONTEXT_DOCUMENT_ROOT = /opt/apache-httpd-2.4.12/cgi-bin/ CONTEXT_PREFIX = /cgi-bin/ DOCUMENT_ROOT = /opt/apache-httpd-2.4.12/htdocs GATEWAY_INTERFACE = CGI/1.1 HTTP_HOST = localhost HTTP_X_MSISDN = test, test2 LD_LIBRARY_PATH = /opt/apache-httpd-2.4.12/lib PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QUERY_STRING = REMOTE_ADDR = 127.0.0.1 REMOTE_PORT = 48128 REQUEST_METHOD = GET REQUEST_SCHEME = http REQUEST_URI = /cgi-bin/test.pl SCRIPT_FILENAME = /opt/apache-httpd-2.4.12/cgi-bin/test.pl SCRIPT_NAME = /cgi-bin/test.pl SERVER_ADDR = 127.0.0.1 SERVER_ADMIN = y...@example.com SERVER_NAME = localhost SERVER_PORT = 80 SERVER_PROTOCOL = HTTP/1.0 SERVER_SIGNATURE = SERVER_SOFTWARE = Apache/2.4.12 (Unix) Connection closed by foreign host. test.pl Description: Perl program
Re: Header with trailing space in field name passed to CGI script
On Tue, 2015-04-14 at 10:09 +0400, George Chelidze wrote: Hello, According to the rfc822#section-3.2, SPACE character is not allowed in the header field name. Be liberal in what you accept. Stripping whitespace between a field name and a colon looks to me like a reasonable thing to do. Unless you're suggesting it could somehow become a security issue? Is there any reason to not ignore a header with the trailing space in the field name and pass it to the CGI environment? Surely rather than ignore it, you'd want to return 400 if you don't accept current behaviour? -- Nick Kew
Re: Header with trailing space in field name passed to CGI script
Hello Nick, Please see my comments below: On 04/14/2015 10:41 AM, Nick Kew wrote: On Tue, 2015-04-14 at 10:09 +0400, George Chelidze wrote: Hello, According to the rfc822#section-3.2, SPACE character is not allowed in the header field name. Be liberal in what you accept. Stripping whitespace between a field name and a colon looks to me like a reasonable thing to do. It looks reasonable to me as well, however there are two things: 1. according to http://tools.ietf.org/html/rfc7230#section-3.2.4, we have: No whitespace is allowed between the header field-name and colon. In the past, differences in the handling of such whitespace have led to security vulnerabilities in request routing and response handling. A server MUST reject any received request message that contains whitespace between a header field-name and colon with a response code of 400 (Bad Request). A proxy MUST remove any such whitespace from a response message before forwarding the message downstream. 2. according to the http://httpd.apache.org/docs/2.4/env.html, the header should be dropped: * A special case are HTTP headers which are passed to CGI scripts and the like via environment variables (see below). They are converted to uppercase and only dashes are replaced with underscores; if the header contains any other (invalid) character, the whole header is silently dropped. See below http://httpd.apache.org/docs/2.4/env.html#fixheader for a workaround. Unless you're suggesting it could somehow become a security issue? I'll try to explain what seems to be an issue for me: In a network with mobile subscribers there is a device called GGSN. We can look at it as an IP gateway for mobile subscribers. In our case, the vendor of GGSN claims, that their device supports HTTP Header Enrichment Feature - it can inject an MSISDN (unique mobile subsciber's identity) header to an HTTP request which is destined to the preconfigured URL (selfcare portal). In case a fake MSISDN header is sent with a request from the user equipment (UE), it's dropped on GGSN. They also insist that their system is developed based on well known standards, and as a header with SPACE in a field name is not a valid header, it's simply passed to the server as is without any action. Suppose the following request has been sent from the UE: GET /cgi-bin/test.pl HTTP/1.0 Host: hostname.for.self.care.portal X-MSISDN : 123456 GGSN will ignore this header as it's not a valid HTTP header and will append a header with a real user's identity so at the end of processing the same request will look like this: GET /cgi-bin/test.pl HTTP/1.0 Host: hostname.for.self.care.portal X-MSISDN : 123456 X-MSISDN: 987654321 After the request hits an apache server, an environment for a CGI script will contain HTTP_X_MSISDN=123456, 987654321 - UE has successfully injected the header which is an issue. Is there any reason to not ignore a header with the trailing space in the field name and pass it to the CGI environment? Surely rather than ignore it, you'd want to return 400 if you don't accept current behaviour? I meant is there any good reason for the server to ignore the SPACE in the header and pass it to the CGI enviroment as if there is no space? I would expect to return 400 according the first document I specified. -- George Chelidze
Re: Header with trailing space in field name passed to CGI script
On Tue, 2015-04-14 at 11:48 +0400, George Chelidze wrote: It looks reasonable to me as well, however there are two things: 1. according to http://tools.ietf.org/html/rfc7230#section-3.2.4, we have: No whitespace is allowed between the header field-name and colon. In the past, differences in the handling of such whitespace have led to security vulnerabilities in request routing and response handling. A server MUST reject any received request message that contains whitespace between a header field-name and colon with a response code of 400 (Bad Request). A proxy MUST remove any such whitespace from a response message before forwarding the message downstream. Damn, I'm getting behind with my RFCs. Yes, that seems to support your position, though it's not entirely clear whether the server or proxy rules should apply (the CGI script is the origin server and never receives the whitespace, while HTTPD's role is as a proxy between the HTTP and CGI protocols). 2. according to the http://httpd.apache.org/docs/2.4/env.html, the header should be dropped: No, that's talking about invalid characters within a header. Not the same as trailing whitespace being stripped. I'll try to explain what seems to be an issue for me: That seems to me pretty clearly a defect in GGSN, whose header it is that's affected. X-MSISDN : 123456 GGSN will ignore this header as it's not a valid HTTP header Yet it MUST return 400 (if we accept that rule applies to HTTPD then it applies equally to $other-agent), and is creating a defect for itself. Whoops! Have you test-driven any other web server or proxy software with this? -- Nick Kew
Re: [RELEASE CANDIDATE] Apache-Test-1.39 RC1
On 04/13/2015 09:23 AM, Steve Hay wrote: On 8 April 2015 at 18:04, Steve Hay steve.m@googlemail.com wrote: On 8 April 2015 at 14:24, Steve Hay steve.m@googlemail.com wrote: Please download, test, and report back on this Apache-Test 1.39 release candidate. http://people.apache.org/~stevehay/Apache-Test-1.39-rc1.tar.gz Success on Windows with VC++ 2010 using perl-5.20.2 with/without LWP and httpd-2.2.29/2.4.10 (4 combinations in all). Please can anyone spend a short while giving this a spin? I will give it a try on Fedora in the afternoon hopefully. Jan Kaluza I've so far only had two reports -- and one of those was mine!
Re: Header with trailing space in field name passed to CGI script
On 04/14/2015 01:21 PM, Nick Kew wrote: Damn, I'm getting behind with my RFCs. Yes, that seems to support your position, though it's not entirely clear whether the server or proxy rules should apply (the CGI script is the origin server and never receives the whitespace, while HTTPD's role is as a proxy between the HTTP and CGI protocols). I would say the server rules should apply as HTTPD's role as a proxy between HTTP and CGI is not pure HTTP proxy it's an HTTP to CGI proxy. No, that's talking about invalid characters within a header. Not the same as trailing whitespace being stripped. My mistake. That seems to me pretty clearly a defect in GGSN, whose header it is that's affected. Do you mean GGSN should identify an invalid header as such and remove it from the request before forwarding it? Yet it MUST return 400 (if we accept that rule applies to HTTPD then it applies equally to $other-agent), and is creating a defect for itself. Whoops! I didn't fully get the idea, sorry. Can you explain in more details? Have you test-driven any other web server or proxy software with this? No I didn't. I'll see what I can do with it. Thanks, -- George Chelidze
unbreak mod_ssl build following removal of chil engine in LibreSSL
I'm trying to update OpenBSD's port of Apache HTTPD to the 2.4 series. This is the first in a small series of diffs from the OpenBSD ports tree. I hope OpenBSD will eventually be able to drop its custom patches. If you're at ApacheCon in Austin this week please feel free to talk to me in person. mod_ssl unconditionally uses a macro related to the chil engine, which has been removed from LibreSSL. Guard use of this macro based on its presence. Patch by Stuart Henderson, obtained by me from the OpenBSD ports tree. http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/www/apache-httpd/patches/patch-modules_ssl_ssl_engine_init_c?rev=1.1content-type=text/x-cvsweb-markup Index: modules/ssl/ssl_engine_init.c === --- modules/ssl/ssl_engine_init.c (revision 1673296) +++ modules/ssl/ssl_engine_init.c (working copy) @@ -374,9 +374,11 @@ return ssl_die(s); } +#ifdef ENGINE_CTRL_CHIL_SET_FORKCHECK if (strEQ(mc-szCryptoDevice, chil)) { ENGINE_ctrl(e, ENGINE_CTRL_CHIL_SET_FORKCHECK, 1, 0, 0); } +#endif if (!ENGINE_set_default(e, ENGINE_METHOD_ALL)) { ap_log_error(APLOG_MARK, APLOG_EMERG, 0, s, APLOGNO(01889)
Extending mod_authz_dbd
Hi, We're preparing the migration to apache 2.4 and we're happy to see that many features now available allow us to put to rest most of our custom developed modules. Looking at mod_authz_dbd, we would like to make a request against a dbase passing different parameters (REQUEST_URI, REQUEST_METHOD, REMOTE_USER, ...) to support a fine-grained ACL mechanism. However, this module doesn't seem to support resolving those variables when preparing the SQL query. Looking at the code, the only one that seems supported is a hard-coded r-user in authz_dbd_group_query: rv = apr_dbd_pvselect(dbd-driver, r-pool, dbd-handle, res, query, 0, r-user, NULL); Is there any historical reason for this? Would you be interested in our contributing time to extend this module to support more generic queries such as (invented query) require sql-query SELECT * FROM foo WHERE user=${REMOTE_USER} AND uri=${REQUEST_URI} AND ${REQUEST_METHOD} in ('GET', 'POST') While browsing for information regarding this point, I only found people asking if this was possible, but no hints if there had been (or will be) plans to do this. The defunct mod_auth_mysql used to propose something similar. i This module is not officially maintained anymore. A possible point of confusion in the module's doc [1] is the phrase saying Since v2.4.8, expressions are supported within the DBD require directives. while the requires in the paragraphs below can't support expressions, as far as I can tell. I may be missing something. Please keep me update if you'd be interested in this generic contribution, which we would be happy to make, rather than forking and maintaining a derived and non-generic patch to this module. If you're interested, I'd appreciate some feedback on how the feature should look like so that it's designed that way before coding it, rather than coding, proposing the contribution and getting feedback it wasn't what x had in mind. Of course, it's understood that your showing interest for this doesn't mean you will end up accepting the contribution. Thanks and KUDOS! Apache 2.4 rocks! -jose
Re: FYI - version checking against libressl - FYI (not yet a bug)
Now fixed in trunk On Apr 12, 2015 4:17 PM, Michael Felt mamf...@gmail.com wrote: I have rebuilt my build systems - basically stripping them of accumulated libraries, and now no OpenSSL installed, but LibreSSL. A basic characteristic of LibreSSL is to remove exposed parts of the API/ABI in order to make the package more secure. Now maybe httpd needs this aspect it is now falling over, in which case - someone much more into httpd code than I explain the need. If it is not really needed this may be a bug. In any case when trying to build httpd-2.4.12 from: ./configure --enable-layout=AIX --with-apr=/opt/bin/apr-1-config --with-apr-util=/opt/bin/apu-1-config --enable-mpms-shared=all --enable-mods-shared=all --disable-lua --enable-ssl --with-ssl=/opt It fails during make with: /opt/build-1/libtool --silent --mode=compile xlc -I/opt/include -qHALT=E -U__STR__ -D_THREAD_SAFE -D_USE_IRS -D_LARGEF/httpd-2.4.12/os/unix -I/data/prj/apache/httpd/httpd-2.4.12/include -I/opt/include/apr-1 -I/opt/include -I/data/prj/apache/httpd/htpd/httpd-2.4.12/modules/cache -I/data/prj/apache/httpd/httpd-2.4.12/modules/core -I/data/prj/apache/httpd/httpd-2.4.12/modules/datadules/filters -I/data/prj/apache/httpd/httpd-2.4.12/modules/ldap -I/data/prj/apache/httpd/httpd-2.4.12/server -I/data/prj/apache/htapache/httpd/httpd-2.4.12/modules/lua -I/data/prj/apache/httpd/httpd-2.4.12/modules/proxy -I/data/prj/apache/httpd/httpd-2.4.12/mod.4.12/modules/ssl -I/data/prj/apache/httpd/httpd-2.4.12/modules/test -I/data/prj/apache/httpd/httpd-2.4.12/server -I/data/prj/apacha/prj/apache/httpd/httpd-2.4.12/modules/dav/main -I/data/prj/apache/httpd/httpd-2.4.12/modules/generators -I/data/prj/apache/httpd/sl_engine_init.c touch ssl_engine_init.slo ssl_engine_init.c, line 357.28: 1506-045 (S) Undeclared identifier ENGINE_CTRL_CHIL_SET_FORKCHECK. make: 1254-004 The error code from the last command is 1. Note: if I leave out the --enable-ssl --with-ssl=/opt the package builds as expected. On Fri, Apr 10, 2015 at 2:26 PM, Michael Felt mamf...@gmail.com wrote: I am experimenting with libressl - and just thought I would mention an error message I am getting with regard to openssl compatibility And if this has already been reported - please ignore - and accept my apologies. I have not scanned the maillist for a previous report. ±±± configure: WARNING: Your APR does not include SSL/EVP support. To enable it: configure --with-crypto configure: WARNING: OpenSSL libraries are unusable Now there are differences between OpenSSL and LibreSSL - so you may need to be thinking about different tests for testing OpenSSL API suitability. ±± From config.log - some additional feedback. configure:25683: checking for OpenSSL version = 0.9.8a configure:25702: xlc -c -I/opt/include -I/opt/buildaix/include -O2 -I/opt/include -L/opt/lib -qHALT=E -I/op t/include -U__STR__ -D_THREAD_SAFE -D_USE_IRS -D_LARGEFILE64_SOURCE conftest.c 5 configure:25702: $? = 0 configure:25703: result: OK ... configure:25772: checking for openssl/engine.h configure:25772: result: yes configure:25785: checking for SSLeay_version configure:25785: xlc -o conftest -I/opt/include -I/opt/buildaix/include -O2 -I/opt/include -L/opt/lib -qHAL T=E -I/opt/include -U__STR__ -D_THREAD_SAFE -D_USE_IRS -D_LARGEFILE64_SOURCE -Wl,-brtl conftest.c -lssl -l crypto -lpthread 5 ld: 0711-317 ERROR: Undefined symbol: .SSLeay_version | } configure:25785: result: no configure:25785: checking for SSL_CTX_new configure:25785: xlc -o conftest -I/opt/include -I/opt/buildaix/include -O2 -I/opt/include -L/opt/lib -qHAL T=E -I/opt/include -U__STR__ -D_THREAD_SAFE -D_USE_IRS -D_LARGEFILE64_SOURCE -Wl,-brtl conftest.c -lssl -l crypto -lpthread 5 ld: 0711-317 ERROR: Undefined symbol: .SSL_CTX_new ... | } configure:25785: result: no configure:25799: checking for ENGINE_init configure:25799: xlc -o conftest -I/opt/include -I/opt/buildaix/include -O2 -I/opt/include -L/opt/lib -qHAL T=E -I/opt/include -U__STR__ -D_THREAD_SAFE -D_USE_IRS -D_LARGEFILE64_SOURCE -Wl,-brtl conftest.c -lssl -l crypto -lpthread 5 ld: 0711-317 ERROR: Undefined symbol: .ENGINE_init ... | } configure:25799: result: no configure:25799: checking for ENGINE_load_builtin_engines configure:25799: xlc -o conftest -I/opt/include -I/opt/buildaix/include -O2 -I/opt/include -L/opt/lib -qHAL T=E -I/opt/include -U__STR__ -D_THREAD_SAFE -D_USE_IRS -D_LARGEFILE64_SOURCE -Wl,-brtl conftest.c -lssl -l crypto -lpthread 5 ld: 0711-317 ERROR: Undefined symbol: .ENGINE_load_builtin_engines ... | } configure:25799: result: no configure:25809: WARNING: OpenSSL libraries are unusable configure:25822: result: yes
Re: unbreak mod_ssl build following removal of chil engine in LibreSSL
On Tue, Apr 14, 2015 at 9:53 AM, Stefan Sperling s...@apache.org wrote: ENGINE_CTRL_CHIL_SET_FORKCHECK Thanks Stefan and Stuart, committed verbatim in r1673455 and will propose for backport soon. I'd expect it in 2.4.13.
Photos at ApacheCon
httpd developers, If you're at ApacheCon, please try to drop by the registration desk at 3:50 for a photo of the httpd committers with The Feather Context - https://www.flickr.com/photos/iamamoose/63963722/ We're going to take several photos with the feather to recreate this photo from 1998, with an updated httpd dev team. Thanks! -- Rich Bowen - rbo...@rcbowen.com - @rbowen http://apachecon.com/ - @apachecon
Re: svn commit: r1673470 - /httpd/httpd/trunk/docs/manual/mod/mod_deflate.xml
Am 14.04.2015 um 19:06 schrieb rbo...@apache.org: Author: rbowen Date: Tue Apr 14 17:06:10 2015 New Revision: 1673470 URL: http://svn.apache.org/r1673470 Log: Add pre-compressed content recipe, as per https://bz.apache.org/bugzilla/show_bug.cgi?id=51336 Modified: httpd/httpd/trunk/docs/manual/mod/mod_deflate.xml Modified: httpd/httpd/trunk/docs/manual/mod/mod_deflate.xml URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/manual/mod/mod_deflate.xml?rev=1673470r1=1673469r2=1673470view=diff == --- httpd/httpd/trunk/docs/manual/mod/mod_deflate.xml (original) +++ httpd/httpd/trunk/docs/manual/mod/mod_deflate.xml Tue Apr 14 17:06:10 2015 @@ -178,6 +178,51 @@ SetEnvIfNoCase Request_URI \.(?:gif|jpe? /example /section +section id=precompressedtitleServing pre-compressed +content/title + +pSince modulemod_deflate/module re-compresses content each +time a request is made, some performance benefit can be derived by +pre-compressing the content and telling mod_deflate to serve them +without re-compressing them. This may be accomplished using a +configuration like the following:/p + +highlight language=config + lt;IfModule mod_headers.cgt; +# Serve gzip compressed CSS files if they exist +# and the client accepts gzip. +RewriteCond %{HTTP:Accept-encoding} gzip +RewriteCond %{REQUEST_FILENAME}\.gz -s +RewriteRule ^(.*)\.css $1\.css\.gz [QSA] + +# Serve gzip compressed JS files if they exist +# and the client accepts gzip. +RewriteCond %{HTTP:Accept-encoding} gzip +RewriteCond %{REQUEST_FILENAME}\.gz -s +RewriteRule ^(.*)\.js $1\.js\.gz [QSA] + + +# Serve correct content types, and prevent mod_deflate double gzip. +RewriteRule \.css\.gz$ - [T=text/css,E=no-gzip:1] +RewriteRule \.js\.gz$ - [T=text/javascript,E=no-gzip:1] + + +lt;FilesMatch (\.js\.gz|\.css\.gz)$gt; + # Serve correct encoding type. + Header append Content-Encoding gzip + + # Force proxies to cache gzipped amp; + # non-gzipped css/js files separately. + Header append Vary Accept-Encoding +lt;/FilesMatchgt; + lt;/IfModulegt; +/IfModule + + +/highlight + +/section + directivesynopsis nameDeflateFilterNote/name descriptionPlaces the compression ratio in a note for logging/description AFAIK mod_rewrite automatically adds headers from RewriteCond to the Vary header as long as no NV flag is used. So the Header append Vary Accept-Encoding part should not be necessary. Thanks for putting effort into documenting the recipe. Regards, Rainer
Re: Header with trailing space in field name passed to CGI script
Have you test-driven any other web server or proxy software with this? A quick update: I have made some tests with lighttpd/1.4.31 and below are the results: 1. Trailing SPACE between the field name and the colon is translated to the underscore in CGI environment: so for X-MSISDN : test you get HTTP_X_MSISDN_=test in the environment. 2. Invalid HTTP Version is handled differently: with apache 2.4.12 GET / HTTP Host: localhost HTTP/1.1 200 OK Date: Tue, 14 Apr 2015 12:51:45 GMT Server: Apache/2.4.12 (Unix) Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT ETag: 2d-432a5e4a73a80 Accept-Ranges: bytes Content-Length: 45 Connection: close Content-Type: text/html with lighttpd 1.4.31 GET / HTTP Host: localhost HTTP/1.0 400 Bad Request Content-Type: text/html Content-Length: 349 Connection: close Date: Tue, 14 Apr 2015 12:52:33 GMT Server: lighttpd/1.4.31 Thanks, -- George Chelidze
Re: Photos at ApacheCon
A huge thank you to those who managed to show up on such short notice. The photos are at https://www.flickr.com/photos/rbowen/sets/72157651553317030/ --Rich On 04/14/2015 03:32 PM, Rich Bowen wrote: httpd developers, If you're at ApacheCon, please try to drop by the registration desk at 3:50 for a photo of the httpd committers with The Feather Context - https://www.flickr.com/photos/iamamoose/63963722/ We're going to take several photos with the feather to recreate this photo from 1998, with an updated httpd dev team. Thanks! -- Rich Bowen - rbo...@rcbowen.com - @rbowen http://apachecon.com/ - @apachecon