Bug#853981: apache2-bin: mod_http2 together with mod_ruid2 breaks the server

[Julian Gilbey]

On Sun, Feb 05, 2017 at 01:59:56PM +0100, Stefan Fritsch wrote:
> On Thursday, 2 February 2017 18:56:38 CET Julian Gilbey wrote:
> > [Thu Feb 02 18:14:44.630796 2017] [core:notice] [pid 3650] AH00052: child
> > pid 3696 exit signal Aborted (6)
> 
> Please follow the instructions in /usr/share/doc/apache2/README.backtrace and 
> add a backtrace to this report. Thanks.

Oops, I forgot to stop the script command.  Here's a better gdb.out.

   Julian
Script started on Wed 08 Feb 2017 21:21:47 GMT
root@redfield:/etc/apache2# strace -p f -p /10047
strace: Process 10047 attached with 27 threads
[pid 10100] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10101] epoll_wait(9,  
[pid 10099] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 24, NULL 
[pid 10098] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 23, NULL 
[pid 10099] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10098] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10099] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10098] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10097] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 22, NULL 
[pid 10096] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 21, NULL 
[pid 10097] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10096] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10097] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10096] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10095] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 20, NULL 
[pid 10094] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 19, NULL 
[pid 10095] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10094] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10095] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10094] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10093] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 18, NULL 
[pid 10092] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 17, NULL 
[pid 10093] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10092] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10093] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10092] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10091] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 16, NULL 
[pid 10089] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 15, NULL 
[pid 10091] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10089] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10091] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10089] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10087] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 14, NULL 
[pid 10085] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 13, NULL 
[pid 10087] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10085] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10087] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10085] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10083] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 12, NULL 
[pid 10081] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 11, NULL 
[pid 10083] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10081] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10083] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10081] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10079] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 10, NULL 
[pid 10077] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 9, NULL 
[pid 10079] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10077] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10079] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10077] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10075] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 8, NULL 
[pid 10073] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 7, NULL 
[pid 10075] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10073] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10075] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10073] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10071] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 6, NULL 
[pid 10069] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 5, NULL 
[pid 10071] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10069] <... futex resumed> )   = -1 EAGAIN (Resource temporarily 
unavailable)
[pid 10071] futex(0x7f913cd4867c, FUTEX_WAIT_PRIVATE, 25, NULL 
[pid 10069] futex(0x7f913cd4867c, 

Bug#853981: apache2-bin: mod_http2 together with mod_ruid2 breaks the server

[Julian Gilbey]

On Sun, Feb 05, 2017 at 01:59:56PM +0100, Stefan Fritsch wrote:
> On Thursday, 2 February 2017 18:56:38 CET Julian Gilbey wrote:
> > [Thu Feb 02 18:14:44.630796 2017] [core:notice] [pid 3650] AH00052: child
> > pid 3696 exit signal Aborted (6)
> 
> Please follow the instructions in /usr/share/doc/apache2/README.backtrace and 
> add a backtrace to this report. Thanks.

Hi Stefan,

I've done as best as I can for this.  Interestingly, when I started 
again with an absolutely clean installation of apache2, with the only
changes from the default installation being:

* adding the CoreDumpDirectory directive to apache2.conf
* enabling the ruid2 module (but not the http2 module)

I still obtained the same failing behaviour (but not a crash)

This is the /var/log/apache2/error.log message:

[Wed Feb 08 21:15:03.829771 2017] [core:notice] [pid 6416:tid 140261767853248] 
AH00052: child pid 9982 exit signal Aborted (6)

Running gdb on process 6416 gave essentially nothing; see the attached
gdb2.out.  (I pressed c after the page had failed to load, then did
ctrl-c to exit.  Nothing displayed in the gdb trace.)

More interesting was the effect of attaching to the child process: see
the gdb.out attached.

I also did an strace on the child process; that is in strace.out.

What is very interesting is that I have had mod_ruid2 on another
machine (also running testing) and it hasn't had any problems.

It certainly looks as though the bug is in that module.

Best wishes,

   Julian
Script started on Wed 08 Feb 2017 21:14:29 GMT
root@redfield:~# gddb -p 6416
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 6416
Reading symbols from target:/usr/sbin/apache2...Reading symbols from 
/usr/lib/debug/.build-id/42/8cd0f45f69360d47886da802ed345070c9bc00.debug...done.
done.
Reading symbols from target:/lib/x86_64-linux-gnu/libpcre.so.3...(no debugging 
symbols found)...done.
Reading symbols from 
target:/usr/lib/x86_64-linux-gnu/libaprutil-1.so.0...Reading symbols from 
/usr/lib/debug/.build-id/27/4bb6314ac13ddc5124af98cc9d95eff07fe9d1.debug...done.
done.
Reading symbols from target:/usr/lib/x86_64-linux-gnu/libapr-1.so.0...Reading 
symbols from 
/usr/lib/debug/.build-id/e4/34c8be0266b67f2a76aeba76ff6231a64d3732.debug...done.
done.
Reading symbols from target:/lib/x86_64-linux-gnu/libpthread.so.0...Reading 
symbols from 
/usr/lib/debug/.build-id/75/b2a574fa9c03e43b58f53b424b1daec1211862.debug...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from target:/lib/x86_64-linux-gnu/libc.so.6...Reading symbols 
from 
/usr/lib/debug/.build-id/4b/9cc30ba41f027a0dca6cd877f59f0db38f4025.debug...done.
done.
Reading symbols from target:/lib/x86_64-linux-gnu/libuuid.so.1...(no debugging 
symbols found)...done.
Reading symbols from target:/lib/x86_64-linux-gnu/librt.so.1...Reading symbols 
from 
/usr/lib/debug/.build-id/93/5f7cac8894edac152c05cf83b2decf1097920e.debug...done.
done.
Reading symbols from target:/lib/x86_64-linux-gnu/libcrypt.so.1...Reading 
symbols from 
/usr/lib/debug/.build-id/50/88d4df6ba5d1a3184f19d7dbac7ed53de3871b.debug...done.
done.
Reading symbols from target:/lib/x86_64-linux-gnu/libdl.so.2...Reading symbols 
from 
/usr/lib/debug/.build-id/e4/8bb27b88670405041a12eefef9ef586f6e1533.debug...done.
done.
Reading symbols from target:/lib/x86_64-linux-gnu/libexpat.so.1...(no debugging 
symbols found)...done.
Reading symbols from target:/lib64/ld-linux-x86-64.so.2...Reading symbols from 
/usr/lib/debug/.build-id/09/5935d2da92389e2991f2b56d14dab9e6978696.debug...done.
done.
Reading symbols from target:/lib/x86_64-linux-gnu/libnss_compat.so.2...Reading 
symbols from 
/usr/lib/debug/.build-id/ce/a6be6f04a6d00a4a6e5fdc8bfb22e1c8106005.debug...done.
done.
Reading symbols from target:/lib/x86_64-linux-gnu/libnsl.so.1...Reading symbols 
from 
/usr/lib/debug/.build-id/6b/809180d68bae984119c3a606fad0c59baeae67.debug...done.
done.
Reading symbols from target:/lib/x86_64-linux-gnu/libnss_nis.so.2...Reading 
symbols from 
/usr/lib/debug/.build-id/05/99d07417db4e3a16202c6c1003016e33d347ef.debug...done.
done.
Reading symbols from target:/lib/x86_64-linux-gnu/libnss_files.so.2...Reading 
symbols from 

Bug#853981: apache2-bin: mod_http2 together with mod_ruid2 breaks the server

[Stefan Fritsch]

On Thursday, 2 February 2017 18:56:38 CET Julian Gilbey wrote:
> [Thu Feb 02 18:14:44.630796 2017] [core:notice] [pid 3650] AH00052: child
> pid 3696 exit signal Aborted (6)

Please follow the instructions in /usr/share/doc/apache2/README.backtrace and 
add a backtrace to this report. Thanks.

Cheers,
Stefan

Bug#853981: apache2-bin: mod_http2 together with mod_ruid2 breaks the server

[Julian Gilbey]

Package: apache2-bin, libapache2-mod-ruid2
Version: 2.4.25-3, 0.9.8-3
Severity: normal

I enabled the http2 and ruid2 modules.  I have had the ruid2 module
running fine for ages without a problem, and just tried enabling
http2.  I left http2 switched off by explicitly writing "Protocols
http/1.1" in apache2.conf (the same behaviour happens when I write
"Protocols h2 h2c http/1.1"), and any attempt to connect to my server
yields lots of error messages of the form:

[Thu Feb 02 18:14:44.630796 2017] [core:notice] [pid 3650] AH00052: child pid 
3696 exit signal Aborted (6)

and my site simply fails to load on a browser.

Disabling ruid2 and enabling http2 allows the server to run fine.

I'm not sure which module is at fault.

Best wishes,

   Julian

-- Package-specific info:

-- System Information:
Debian Release: 9.0
  APT prefers jessie
  APT policy: (500, 'jessie'), (500, 'testing'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.9.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) (ignored: LC_ALL 
set to en_GB.UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages apache2-bin depends on:
ii  libapr1  1.5.2-5
ii  libaprutil1  1.5.4-3
ii  libaprutil1-dbd-sqlite3  1.5.4-3
ii  libaprutil1-ldap 1.5.4-3
ii  libc62.24-8
ii  libldap-2.4-22.4.44+dfsg-3
ii  liblua5.2-0  5.2.4-1.1+b1
ii  libnghttp2-141.18.1-1
ii  libpcre3 2:8.39-2
ii  libssl1.0.2  1.0.2j-5
ii  libxml2  2.9.4+dfsg1-2.2
pn  perl:any 
ii  zlib1g   1:1.2.8.dfsg-4

apache2-bin recommends no packages.

Versions of packages apache2-bin suggests:
ii  apache2-doc  2.4.25-3
pn  apache2-suexec-pristine | apache2-suexec-custom  
ii  elinks [www-browser] 0.12~pre6-12
ii  epiphany-browser [www-browser]   3.22.5-1
ii  firefox-esr [www-browser]45.7.0esr-1
ii  google-chrome-stable [www-browser]   56.0.2924.87-1
ii  links [www-browser]  2.14-2
ii  links2 [www-browser] 2.14-2
ii  lynx [www-browser]   2.8.9dev11-1
ii  w3m [www-browser]0.5.3-34

Versions of packages apache2 depends on:
ii  apache2-data 2.4.25-3
ii  apache2-utils2.4.25-3
ii  dpkg 1.18.18
ii  init-system-helpers  1.47
ii  lsb-base 9.20161125
ii  mime-support 3.60
ii  perl 5.24.1-1
pn  perl:any 
ii  procps   2:3.3.12-3

Versions of packages apache2 recommends:
ii  ssl-cert  1.0.38

Versions of packages apache2 suggests:
ii  apache2-doc  2.4.25-3
pn  apache2-suexec-pristine | apache2-suexec-custom  
ii  elinks [www-browser] 0.12~pre6-12
ii  epiphany-browser [www-browser]   3.22.5-1
ii  firefox-esr [www-browser]45.7.0esr-1
ii  google-chrome-stable [www-browser]   56.0.2924.87-1
ii  links [www-browser]  2.14-2
ii  links2 [www-browser] 2.14-2
ii  lynx [www-browser]   2.8.9dev11-1
ii  w3m [www-browser]0.5.3-34

Versions of packages apache2-bin is related to:
ii  apache2  2.4.25-3
ii  apache2-bin  2.4.25-3

-- no debconf information