Re: help needed to complete regression fix for apache2 Bug#858373
Hi, sorry for the delay. Gmail filed this one into spam :-( Unfortunately, I don't have access to that environment anymore to confirm. I'll pass this on to the folks that do so hopefully they can. My recollection from this issue was that I'd tested it against different package versions and the 400 ErrorDocuments had worked beforehand (we used them for ModSec types of things primarily, and I'm confident that mode was working well before hand and after), though possibly not in that particular protocol error context. I vaguely recall having issues reproducing a working ErrorDocument with non-cgi methods in that protocol error mode style test as well, but I don't recall if rhat was only in the newer versions of the software that I had been testing with or true before that update as well. Anyways, thanks much for following up. Sorry I don't have more info to offer at the moment. Cheers, Brian On Fri, Jul 21, 2017, 08:44 Antoine Beaupré <anar...@orangeseeds.org> wrote: > TL;DR: New proposed package (deb7u11) doesn't ctually show a new > regression, please test: > > > https://people.debian.org/~anarcat/debian/wheezy-lts/apache2_2.2.22-13+deb7u11_amd64.changes > > In particular, Brian Kroth: are you *sure* you had that ErrorDocument > 400 working in apache2_2.2.22-13+deb7u7 (ie. before the DLA-841-1 > upload)? In my tests, it didn't actually work at all. It wouldn't > trigger a segfault, but the CGI script wouldn't get called either. In > the above package, we don't segfault anymore, but we yield a 400 + 500 > error message (because the ErrorDocument fails). The solution, here, is > obviously to update to a later Apache version (e.g. update to jessie, > really) to get that functionality working, from my perspective. > > More technical details follow. > > On 2017-07-21 09:24:00, Stefan Fritsch wrote: > > Hi Antoine, > > > > On Wednesday, 19 July 2017 15:45:20 CEST Antoine Beaupre wrote: > >> As I mentioned in the #858373 bug report, I started looking at fixing > >> the regression introduced by the 2.2.22-13+deb7u8 upload, part of > >> DLA-841-1. The problem occurs when a CGI(d) ErrorDocument is configured > >> to handle 400 error messages that can be triggered with a simple "GET / > >> HTTP/1.0\n\n". Such a request segfaults Apache in Wheezy right now. > > > >> Unfortunately, re-introducing the protocol initialization code isn't > >> sufficient: it does fix the segfaults, but the ErrorDocument handling is > >> not quite working yet. Instead of seeing the output of the > >> ErrorDocument, after 10 seconds, I get the raw 400 message, doubled with > >> a 500 error document warning: > > > >> Note that I have also tried to see if sending "\r\n" instead of just > >> "\n" in my "hello world" example would work around the issue: it > >> doesn't, unfortunately. > >> > >> I am at a loss as where to go from here, to be honest. The patch > >> (attached) at least fixes the segfault, which resolves the primary issue > >> at hand here (DoS by crashing processes!) but it would be nice to > >> actually fix the ErrorDocument as well.. > > > > This sounds familiar. Maybe it's simply broken in 2.2.22. Can you > compare with > > 2.2.22-13+deb7u7 if that bug has been there already? > > Well, the problem is - how do I reproduce this? I can't generate the > same 400 error message in deb7u7 (I tried!) with the previous techniques > because the new request handling code isn't there. That is, the > following query just works: > > # printf "GET / HTTP/1.0\n\n" | nc localhost 80 | head -1 > HTTP/1.1 200 OK > > > Furthermore, generating a 400 error, when it works in deb7u7, doesn't > trigger the ErrorDocument - not sure why: > > # printf "G ET / HTTP/1.0\r\n\r\n" | nc localhost 80 > HTTP/1.1 400 Bad Request > Date: Fri, 21 Jul 2017 13:40:48 GMT > Server: Apache/2.2.22 (Debian) > Vary: Accept-Encoding > Content-Length: 302 > Connection: close > Content-Type: text/html; charset=iso-8859-1 > > > > 400 Bad Request > > Bad Request > Your browser sent a request that this server could not understand. > > > Apache/2.2.22 (Debian) Server at wheezy.raw Port 80 > > > Logs show the following: > > [Fri Jul 21 13:40:48 2017] [error] [client 127.0.0.1] Invalid URI in > request G ET / HTTP/1.0 > > ... whether or not the 400 ErrorDocument directive is present. Notice > how the ErrorDocument isn't triggered at all here. > > Of course, a 404 ErrorDocument still works correctly: > > # printf "GET /wtf HTTP/1.0\r\n\r\n" | nc localhost 80 > HTTP/1.1 404 Not Found > Date: Fri, 21 Ju
Bug#858373: apache2: segfaults upon recieving bad request when using worker/event mpm and cgid errordoc
Package: apache2.2-common Version: 2.2.22-13+deb7u8 Severity: normal Tags: security Dear Maintainer, We have some websites running on Debian Wheezy, so still using Apache 2.2.22, that are configured either in Worker or Event MPM (so are using mod_cgid in what follows), and have a custom "ErrorDocument 400" directive that points at a perl script for providing custom ModSecurity error. I haven't dug up an older version of the package from snapshots to confirm this, but I think that since the recently backported HttpProtocolOptions directive to that version (BTW, where was that announced - I had to run strings on the binary to find it), I've been seeing a lot of segfault/coredumps registered in the Apache error logs. After some analysis, I've found that I can reproduce the error with a fairly trivial shell command: # echo -ne "GET / HTTP/1.0\n" | nc $some_website 80 From the coredump, I was able to find that this line (1371) in the cgid_handler() code in the modules/generators/mod_cgid.c source file has a null pointer issue on the r->protocol field: is_included = !strcmp(r->protocol, "INCLUDED"); Seems like a bit of a security issue to me. No combination of adjustments to the HttpProtocolOptions directive seemed to help from what I could see. I also haven't been able to reproduce this issue on a Prefork MPM backend webserver. Varying the details of the perl ErrorDocument script's implementation don't appear to help either (eg: it still occurs even with a simple hello world script). The error still occurs if I disable ModSecurity, but leave the ErrorDocument for 400 messages. Let me know if you need any more details or have any questions. Thanks, Brian -- Package-specific info: List of /etc/apache2/mods-enabled/*.load: alias auth_basic authn_file authz_default authz_groupfile authz_host authz_user autoindex cgid deflate dir env expires headers include info mime mod-security negotiation reqtimeout rewrite rpaf setenvif shib2 ssl status unique_id userdir* (A * means that the .conf file for that module is not enabled in /etc/apache2/mods-enabled/) -- System Information: Debian Release: 7.11 APT prefers oldstable APT policy: (500, 'oldstable') Architecture: amd64 (x86_64) Kernel: Linux 3.16.0-0.bpo.4-amd64 (SMP w/1 CPU core) Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages apache2 depends on: ii apache2-mpm-worker 2.2.22-13+deb7u8 ii apache2.2-common2.2.22-13+deb7u8 apache2 recommends no packages. apache2 suggests no packages. Versions of packages apache2.2-common depends on: ii apache2-utils 2.2.22-13+deb7u8 ii apache2.2-bin 2.2.22-13+deb7u8 ii lsb-base 4.1+Debian8+deb7u1 ii mime-support 3.52-1+deb7u1 ii perl 5.14.2-21+deb7u4 ii procps 1:3.3.3-3 Versions of packages apache2.2-common recommends: pn ssl-cert Versions of packages apache2.2-common suggests: pn apache2-doc pn apache2-suexec | apache2-suexec-custom ii lynx-cur [www-browser] 2.8.8dev.12-2+deb7u1 -- no debconf information
Bug#619408: apache2.2-common: mod_authnz_ldap require directives unrecognized if loaded after mod_authnz_default
Stefan Fritsch s...@sfritsch.de 2011-03-23 23:01:
On Wed, 23 Mar 2011, Brian P Kroth wrote:
In the default configuration mod_authnz_ldap.load is symlinked from
mods-available to mods-enabled but that orders it (lexicographically)
after the symlink to load mod_authnz_default. This causes a number of
ldap specific arguments to the Require definition to be unrecognized and
logged as follows:
[Wed Mar 23 11:04:48 2011] [error] [client 10.10.10.10] access to
/auth failed, reason: unknown require directive:ldap-user bpktest
bpkroth
This message is not produced by apache2's core or any of the default
modules. Therefore I expect that the bug is in a third party module.
To find out which module may be the culprit, enter as root:
grep require: $(cat /proc/XXX/maps |fgrep .so|awk '{print $6}'|sort -u)
Replace XXX with the pid of a running apache process.
Fixed your grep up and did this instead:
# for pid in `pgrep apache2`; do for so in `cat /proc/$pid/maps | fgrep .so |
awk '{ print $6 }' | sort -u`; do strings $so | grep -qi 'unknown require
directive:' echo $so; done; done
Which turned up this:
/usr/lib/apache2/modules/mod_auth_plain.so
This is in this package:
# dpkg -S /usr/lib/apache2/modules/mod_auth_plain.so
libapache2-mod-auth-plain: /usr/lib/apache2/modules/mod_auth_plain.so
Anything else?
Thanks,
Brian
signature.asc
Description: Digital signature
