** Description changed:
+ [Impact]
+ The default apache2.conf causes apache to issue streams of error
+ messages about deadlocks acquiring the SSL session cache lock.
+
+ Users are still reporting seeing this flaw in production (Xenial-based)
+ hosts.
+
+ [Test Case]
+ Reproduction steps TBD. Problem exhibits on high load systems. Verification
will need to be done by those seeing the issue in production.
+
+ [Regression Potential]
+ Since this only changes the config installed by default, it won't impact
existing installations, however behaviors to watch for would be SSL-related or
configuration-related oddnesses.
+
+ [Fix]
+ Backport a fix applied in bionic and newer, that modifies the makes
+ Apache use pthread mutexes by default on Linux, or fctnl on other
+ architectures that lack robust pthread muxexes.
+
+ [Original Report]
OS:
Ubuntu 14.04 LTS
Kernel:
3.13.0-79-generic x86_64
Apache:
2.4.7-1ubuntu4.5
In the default Apache 2.4 config on Ubuntu 14.04 LTS is the following
set in /etc/apache2/apache2.conf:
Mutex file:${APACHE_LOCK_DIR} default
(/debian/config-dir/apache2.conf in
http://archive.ubuntu.com/ubuntu/pool/main/a/apache2/apache2_2.4.7-1ubuntu4.5.debian.tar.gz)
which leads to the following output of "apache2ctl -t -D DUMP_RUN_CFG":
Mutex default: dir="/var/lock/apache2" mechanism=fcntl
This leads constantly to a lot of these warning/emergency messages on a
server with 200 busy worker threads, 100 Requests/s, 300 KB/s:
[Tue Mar 08 16:08:18.596653 2016] [ssl:warn] [pid 8339:tid
140182179256064] (35)Resource deadlock avoided: AH02026: Failed to
acquire SSL session cache lock
[Wed Mar 09 07:09:31.099331 2016] [mpm_worker:emerg] [pid 26526:tid
139668485949184] (35)Resource deadlock avoided: AH00273:
apr_proc_mutex_lock failed. Attempting to shutdown process gracefully.
Solution (as suggested by Yann Ylavic from Apache):
Commenting (removing) the Mutex directive, which leads to the following
output of "apache2ctl -t -D DUMP_RUN_CFG":
Mutex default: dir="/var/run/apache2/" mechanism=default
Then, there are no error messages anymore.
For the discussion, see the corresponding Apache httpd-users mailing
list thread:
http://httpd.markmail.org/message/c7w5aujfmy2kfazi
(thread subject 'Lots of messages "[ssl:warn] Resource deadlock avoided:
AH02026: Failed to acquire SSL session cache lock"' from 2016-03-08)
-
Here some more information:
# apache2ctl -V
Server version: Apache/2.4.7 (Ubuntu)
Server built: Jul 24 2015 17:25:11
Server's Module Magic Number: 20120211:27
Server loaded: APR 1.5.1-dev, APR-UTIL 1.5.3
Compiled using: APR 1.5.1-dev, APR-UTIL 1.5.3
Architecture: 64-bit
Server MPM: worker
- threaded: yes (fixed thread count)
- forked: yes (variable process count)
+ threaded: yes (fixed thread count)
+ forked: yes (variable process count)
Server compiled with....
- -D APR_HAS_SENDFILE
- -D APR_HAS_MMAP
- -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
- -D APR_USE_SYSVSEM_SERIALIZE
- -D APR_USE_PTHREAD_SERIALIZE
- -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
- -D APR_HAS_OTHER_CHILD
- -D AP_HAVE_RELIABLE_PIPED_LOGS
- -D DYNAMIC_MODULE_LIMIT=256
- -D HTTPD_ROOT="/etc/apache2"
- -D SUEXEC_BIN="/usr/lib/apache2/suexec"
- -D DEFAULT_PIDLOG="/var/run/apache2.pid"
- -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
- -D DEFAULT_ERRORLOG="logs/error_log"
- -D AP_TYPES_CONFIG_FILE="mime.types"
- -D SERVER_CONFIG_FILE="apache2.conf"
+ -D APR_HAS_SENDFILE
+ -D APR_HAS_MMAP
+ -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
+ -D APR_USE_SYSVSEM_SERIALIZE
+ -D APR_USE_PTHREAD_SERIALIZE
+ -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
+ -D APR_HAS_OTHER_CHILD
+ -D AP_HAVE_RELIABLE_PIPED_LOGS
+ -D DYNAMIC_MODULE_LIMIT=256
+ -D HTTPD_ROOT="/etc/apache2"
+ -D SUEXEC_BIN="/usr/lib/apache2/suexec"
+ -D DEFAULT_PIDLOG="/var/run/apache2.pid"
+ -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
+ -D DEFAULT_ERRORLOG="logs/error_log"
+ -D AP_TYPES_CONFIG_FILE="mime.types"
+ -D SERVER_CONFIG_FILE="apache2.conf"
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1565744
Title:
"Mutex file:${APACHE_LOCK_DIR} default" should be disabled by default
on Linux because it leads to errors
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1565744/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
