Am 09.08.2019 um 15:40 schrieb Daniel Ruggeri:
Hi, all;
Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/
I would like to call a VOTE over the next few days to release this
candidate tarball as 2.4.41:
[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.
The computed digests of the tarball up for vote are:
sha1: b713e835aa7cde823a4b7f8e3463164f3d9fe63e *httpd-2.4.41.tar.gz
sha256: 3c0f9663240beb0f008acf3b4501c4f339d7467ee345a36c86c46b4d6f3a5461
*httpd-2.4.41.tar.gz
--
Daniel Ruggeri
+1 to release and thanks a bunch for RM!
Summary: all OK except for
- very few shutdown crashes on Solaris (already observed in 2.4.37, but
then with MPM event when statically linked, now 2 times both with
prefork and shared linking)
- not tested but still expected to happen: problems with prefork plus
mod_ext_filter plus LimitRequestBody on Solaris (not a regression), plus
indicatons that I also saw it on SLES 11 (and then disabled
mod_ext_filter there to let the tests proceed).
Detailed report:
- Sigs and hashes OK
- contents of tarballs identical
- contents of tag and tarballs identical
except for expected deltas
Built on
- Solaris 10 Sparc as 32 Bit Binaries
- SLES 11+12 (64 Bits)
- RHEL 6+7 (64 Bits)
For all platforms built
- with default (shared) and static modules
- with module set reallyall
- using --enable-load-all-modules
- against external APR/APU 1.7.0/1.6.1
plus for SLES 12 and RHEL 7 also against 1.6.5/1.6.1
plus for SLES 12 and RHEL 7 built but not tested using APR trunk head
- using external libraries
- expat 2.2.7
- pcre 8.43
- lua 5.3.5 (compiled with LUA_COMPAT_MODULE)
- distcache 1.5.1
- libxml2 2.9.9
- libnghttp2 1.39.1
- brotli 1.0.7
- curl 7.65.3
- jansson 2.12
and
- openssl 0.9.8zh, 1.0.2, 1.0.2s, 1.0.1e, 1.0.1k, 1.1.1, 1.1.1c plus
patches (head of a few days ago)
- Tool chain:
- platform gcc except on Solaris
(gcc 8.3.0 Solaris 10)
- CFLAGS: -O2 -g -Wall -fno-strict-aliasing
- on Solaris additionally -mpcu=v9, -D_XOPEN_SOURCE,
-D_XOPEN_SOURCE_EXTENDED=1, -D__EXTENSIONS__
and -D_XPG6
All of the 246 builds succeeded.
- compiler warnings: none
Tested for
- Solaris 10, SLES 11+12, RHEL 6+7
- MPMs prefork, worker, event
- default and static module builds
- log level trace8
- module set reallyall
- for "reallyall" 129 modules plus MPMs
- Perl client bundle build against OpenSSL 1.1.1, 1.1.0i, 1.0.2p and 0.9.8zh
- OpenSSL once linked statically and once as a shared library
Every OpenSSL version in the client tested with every version in the
server, not just the same version.
The total number of test suite runs was 1926 (plus some on Solaris and
RHEL 6 still running, the whole suite hasn't finished yet, but enough to
come up with a clear result).
The following test failures were seen:
a Crashes only on Solaris, this time only with prefork MPM and
dynamically linked builds.
The crash seems to happen only at the end of a process during pchild
clean up and it might be problematic, that the watchdog thread at that
time still exists.
gdb info see at end.
b Tests 4, 8 and 12 of t/modules/buffer.t
Not a regression
Tests 4, 8 and sometimes 12, always line 37
Relatively frequent (645) failures on platforms Solaris 10
and old SLES11, RHEL 6, but not on more modern (and here faster)
SLES 12 and RHEL 7. Happens for all OpenSSL client and server
versions and all link types.
c Various tests in t/apache/expr_string.t
Not a regression.
18 times,
Test numbers : 11, 14, 20, 23, 26, 29
Happens for 18 out of 1926 runs
(3 times on Solaris 10,
otherwise always on RHEL6).
The failure is once on line 101, all others are on line 87.
They happen, when the error_log contents are checked.
Could be due to logs written to NFS.
d Test 5 in t/modules/dav.t line 69:
Not a regression.
5 times: once RHEL 6, twice RHEL 7 and twice SLES 12
Creation, modified and now times not in the correct order.
This seems to be a system issue, all tests done on NFS,
many tested on virtualized guests.
e Test 54 in t/modules/cgi.t line 232:
Not a regression
3 times once Solaris
Test checks log contents. Could be false positive due to
logs written to NFS.
Regards,
Rainer
GDB info (sporadic) Solaris shutdown crahes:
#0 0xff07f17c in allocator_free (node=0x1bd8, allocator=0x333530) at
/path/to/sources/apr/1.7.x/1.7.0/apr-1.7.0/memory/unix/apr_pools.c:486
freelist = 0x0
max_index = 1574824
max_free_index = 512
next = 0x1bd8
index = 5201872
current_free_index = 4284842517
#1 apr_pool_destroy (pool=0x338068) at
/path/to/sources/apr/1.7.x/1.7.0/apr-1.7.0/memory/unix/apr_pools.c:1043
active = <optimized out>
allocator = 0x333530
#2 0xfed629d0 in clean_child_exit (code=7) at
/shared/build/dev/httpd/sources/httpd/2.4.x/2.4.41/httpd-2.4.41/server/mpm/prefork/prefork.c:227
No locals.
#3 0xfed62f20 in child_main (child_num_arg=<optimized out>,
child_bucket=<optimized out>)
at
/path/to/sources/httpd/2.4.x/2.4.41/httpd-2.4.41/server/mpm/prefork/prefork.c:481
pfd = <optimized out>
thd = 0x3380a8
osthd = 1
ptrans = 0x33a070
allocator = 0x333530
status = <optimized out>
i = <optimized out>
lr = <optimized out>
pollset = 0x3395e8
sbh = 0x3395e0
bucket_alloc = <optimized out>
last_poll_idx = <optimized out>
lockfile = <optimized out>
#4 0xfed635fc in make_child (s=0xcb1b8, slot=6) at
/path/to/sources/httpd/2.4.x/2.4.41/httpd-2.4.41/server/mpm/prefork/prefork.c:717
bucket = <optimized out>
pid = <optimized out>
#5 0xfed645e0 in perform_idle_server_maintenance (p=<optimized out>) at
/path/to/sources/httpd/2.4.x/2.4.41/httpd-2.4.41/server/mpm/prefork/prefork.c:821
i = <optimized out>
idle_count = <optimized out>
ws = <optimized out>
free_length = <optimized out>
free_slots = {6, 7, 7, 8, -29193744, 3358792, -29193712, 0, 0,
0, 0, 0, 0, 0, 0, -51880160, 0, 7168, -18130688, 0, -15783424, 655472,
-4197456, 356208, 1000,
-29192840, -40916880, 1000, 2000736, -56919604, 0, 1000}
last_non_dead = <optimized out>
total_non_dead = <optimized out>
bucket_kill_child_record = 0
#6 prefork_run (_pconf=<optimized out>, plog=<optimized out>,
s=<optimized out>)
at
/path/to/sources/httpd/2.4.x/2.4.41/httpd-2.4.41/server/mpm/prefork/prefork.c:1014
status = 0
pid = {pid = -1, in = 0xffbff398, out = 0xfe421668
<ap_watchdog_register_callback>, err = 0xffbff3ac}
child_slot = <optimized out>
exitwhy = APR_PROC_EXIT
processed_status = <optimized out>
index = <optimized out>
remaining_children_to_start = 0
i = <optimized out>
#7 0x0003a8e4 in ap_run_mpm (pconf=0xa6040, plog=0xccd10, s=0xcb1b8) at
/path/to/sources/httpd/2.4.x/2.4.41/httpd-2.4.41/server/mpm_common.c:94
pHook = <optimized out>
n = 0
rv = -1
It seems to crash in the allocator cleanup for the pchild pool:
(gdb) print *allocator
$1 = {max_index = 1574824, max_free_index = 512, current_free_index =
4294967295, mutex = 0x0, owner = 0x0, free = {0x4e24a8, 0x0 <repeats 18
times>, 0x333528}}
(gdb) up
#1 apr_pool_destroy (pool=0x338068) at
/path/to/sources/apr/1.7.x/1.7.0/apr-1.7.0/memory/unix/apr_pools.c:1043
1043 allocator_free(allocator, active);
(gdb) print *pool
$2 = {parent = 0x0, child = 0x0, sibling = 0x335c48, ref = 0xa6044,
cleanups = 0x0, free_cleanups = 0x0, allocator = 0x333530, subprocesses
= 0x0,
abort_fn = 0x324d0 <abort_on_oom>, user_data = 0x0, tag = 0xfed6a430
"pchild", active = 0x364100, self = 0x338050, self_first_avail =
0x3380a8 "", pre_cleanups = 0x0}
A second thread s still running, namely the watchdog thread: I guess
that thread should have finished first. It gets the SIGTERM, but that is
maybe triggered by the first thread:
#0 0xfee42480 in mutex_lock_impl () from /lib/libc.so.1
#1 0xfecd60c4 in __gthread_mutex_lock (__mutex=0xfece6fd0
<object_mutex>) at gthr-default.h:748
#2 __deregister_frame_info_bases (begin=0xfece7000) at
../../../gcc-8.3.0/libgcc/unwind-dw2-fde.c:211
#3 0xfecd0c84 in __do_global_dtors_aux ()
from
/shared/build/dev/httpd/install/httpd/2.4.x/2.4.41/solaris10.sparc-1.7.0-1.6.1-1.1.0k-1-shared-reallyall-default/modules/mod_authn_dbd.so
#4 0xfecd6888 in _fini () from
/shared/build/dev/httpd/install/httpd/2.4.x/2.4.41/solaris10.sparc-1.7.0-1.6.1-1.1.0k-1-shared-reallyall-default/modules/mod_authn_dbd.so
#5 0xff3c5a64 in call_fini () from /lib/ld.so.1
#6 0xff3c5c34 in atexit_fini () from /lib/ld.so.1
#7 0xfedc237c in _exithandle () from /lib/libc.so.1
#8 0xfedb0798 in exit () from /lib/libc.so.1
#9 0xfed62a08 in clean_child_exit (code=0) at
/shared/build/dev/httpd/sources/httpd/2.4.x/2.4.41/httpd-2.4.41/server/mpm/prefork/prefork.c:236
#10 0xfed62a2c in just_die (sig=15) at
/shared/build/dev/httpd/sources/httpd/2.4.x/2.4.41/httpd-2.4.41/server/mpm/prefork/prefork.c:355
#11 <signal handler called>
#12 0xfee4cdc0 in __pollsys () from /lib/libc.so.1
#13 0xfee3cac4 in _pollsys () from /lib/libc.so.1
#14 0xfede8598 in pselect () from /lib/libc.so.1
#15 0xfede8910 in select () from /lib/libc.so.1
#16 0xff08ba8c in apr_sleep (t=100000) at
/shared/build/dev/httpd/sources/apr/1.7.x/1.7.0/apr-1.7.0/time/unix/time.c:246
#17 0xfe422d64 in wd_worker (thread=<optimized out>, data=0x333cc0) at
/shared/build/dev/httpd/sources/httpd/2.4.x/2.4.41/httpd-2.4.41/modules/core/mod_watchdog.c:151
#18 0xff08afe8 in dummy_worker (opaque=0x335d28) at
/shared/build/dev/httpd/sources/apr/1.7.x/1.7.0/apr-1.7.0/threadproc/unix/thread.c:142
#19 0xfee494f8 in _lwp_start () from /lib/libc.so.1
