Bug#905603: Tested on Buster : bug still present
Hi, Following your advice, I tested with Debian buster and the development version of perl and mod_perl and this bug is still present. I filed a report on the mod_perl development mailing list. Benoit
Bug#905650: Tested on Debian buster : bug fixed
Hi, Following your advise I first tested if the bug is still there on Debian testing and it's not : the sequence of event is still the same as I described except that the relocation dependency of libm towards libperl is gone and now libperl unloads properly when mod_perl is unloaded => the PL_check array is fresh clean on the second reload and there is no invalid pointer. So there is a fix that could perhaps be backported in Debian stretch about removing this relocation dependency. I didn't investigate what it could be.
Bug#905650: update
After thinking about it, my proposed patch will not work in this situation: 1. use $[ but don't bring libstdc++ => libperl.so will be removed but not arybase.so 2. on create-config stage, libperl.so is reloaded, PL_check is initialized and points to local checks only 3. the boot section is arybase is executed, but wrap_op_checker will be a no-op because the static ab_old_ck_.. pointer is not null => the op checker is not installed in libperl... It's a tough one. To bring the patch further, libperl should also be forced to stay in memory The correct fix is that all XS modules that register things outside themselves should unregister properly on unload. I have identified 3 XS modules that don't do that: arybase, EV, Params::Classify To their defense, Perl is making it difficult (no API to unset an op checker) To Perl's defense, apache is making it difficult by by doing a config load-unload-reload.
Bug#905679: libparams-classify-perl: XS version causing infinite loop when loaded twice
Package: libparams-classify-perl Version: 0.013-6+b1 Severity: important Tags: patch upstream The XS version of this package registers an op checker by writing directly into PL_check array (see libparams-classify-perl-0.013: lib/Params/Classify.xs line 741) This is not the canonic way of registering a op checker, it should use wrap_op_checker instead. This bug is only causing problem if the BOOT section is executed twice : on the second execution, the infinite loop will form when nxck_entersub will point to myck_entersub (as PL_check[OP_ENTERSUB] already points to myck_entersub). Note: the bug is fixed in version 0.015 (buster). Here is the suggested patch for version 0.013 : --- a/lib/Params/Classify.xs2018-08-03 22:55:45.0 +0200 +++ b/lib/Params/Classify.xs2018-08-03 22:55:45.0 +0200 @@ -738,6 +738,5 @@ newSVpvn_share(rtypemeta->keyword_pv, strlen(rtypemeta->keyword_pv), 0); } - nxck_entersub = PL_check[OP_ENTERSUB]; - PL_check[OP_ENTERSUB] = myck_entersub; + wrap_op_checker(OP_ENTERSUB, myck_entersub, _entersub); } -- System Information: Debian Release: 9.5 APT prefers stable-updates APT policy: (500, 'stable-updates'), (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 4.9.0-7-amd64 (SMP w/1 CPU core) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages libparams-classify-perl depends on: ii libc6 2.24-11+deb9u3 ii perl5.24.1-3+deb9u4 ii perl-base [perlapi-5.24.1] 5.24.1-3+deb9u4 libparams-classify-perl recommends no packages. Versions of packages libparams-classify-perl suggests: pn libscalar-number-perl -- no debconf information
Bug#905650: libapache2-mod-perl2: crash when $[ and XML::LibXML used at the same time
Package: libapache2-mod-perl2 Version: 2.0.10-2 Severity: important Tags: upstream patch -- Package-specific info: -8<-- Start Bug Report 8<-- 1. Problem Description: To reproduce the bug, start from a vanilla Debian server, install apache2, libapache2-mod-perl2 and libxml-libxml-perl. Create a perl file handler.pl with following content : #!/usr/bin/perl use XML::LibXML; $[; 1; Add a directive to /etc/apache2/apache2.conf (or add a file in /etc/apache2/conf-enabled with the same directive) : PerlRequire "/path/to/handler.pl" Apache will crash on startup. === I did an in-depth investigation of this problem. Here is what happens: apache goes through 3 stages on startup : pre-config, destroy-config, create-config In the pre-config stage, all modules are loaded loading mod_perl.so also brings libperl.so and libm.so in the program map XML::LibXML is a XS package that has a dependency on libstc++.so => LibXML.so and libstdc++.so.6 are brought in the program map libstdc++.so has a dependency on libm.so libm.so has a relocation dependency on libperl.so (this is strange but true). => a 'dependency cluster' is created between libstdc++, libm and libperl. The special $[ variable is implemented in the XS library arybase.so, it installs an op checker via wrap_op_checker() in the perl interpreter (see perl-5.24.1: ext/arybase/arybase.xs BOOT section) The address of the op checker is saved in the static PL_check array that resides in libperl.so The op checker itself resides in arybase.so Installing an op checker is a one-way operation, Perl provides no API to remove it. At the end of pre-config stage we have an informal dependency libperl.so(PL_check) -> arybase.so During the destroy-config stage, apache unloads all the modules. For mod_perl this implies removing all XS modules (see libapache2-mod-perl2-2.0.10: src/modules/perl/modperl_interp.c:modperl_interp_destroy()) At this stage the op checker in PL_check points to invalid memory as arybase.so was unloaded! Then apache removes mod_perl itself (see apr-1.5.2: dso/unix/dso.c:dso_cleanup()) Normally, this would also remove libperl.so and libm.so but with the dependency cluster that was created with libstc++, libperl is not removed! During create-config stage, mod_perl is reloaded but not libperl of course. loading arybase.so is differed until the PerlRequire directive is executed => PL_check points to invalid memory. The first thing mod_perl does is load /etc/perl/sitecustomize.pl => it crashes on the first op involving the corrupted op checked. This bug is similar to bug #905603 : a pointer to invalid memory is kept in memory after config-destroy. == The fix that I proposed for bug #905603 will work here too: the crash is avoided by skipping the unloading the XS modules: --- a/src/modules/perl/modperl_interp.c 2018-08-03 20:49:54.0 +0200 +++ b/src/modules/perl/modperl_interp.c 2018-08-03 20:49:54.0 +0200 @@ -122,11 +122,11 @@ MP_TRACE_i(MP_FUNC, "*error - still in use!*"); } -handles = modperl_xs_dl_handles_get(aTHX); +/*handles = modperl_xs_dl_handles_get(aTHX);*/ modperl_perl_destruct(interp->perl); -modperl_xs_dl_handles_close(handles); +/*modperl_xs_dl_handles_close(handles);*/ free(interp); } An ideal solution would be that the faulty XS modules do proper cleanup on unloading but Perl does not make it simpler (no API to remove a op checker). 2. Used Components and their Configuration: *** mod_perl version 2.10 *** using /usr/lib/x86_64-linux-gnu/perl5/5.24/Apache2/BuildConfig.pm *** Makefile.PL options: MP_APR_CONFIG => /usr/bin/apr-config MP_APR_LIB => aprext MP_APXS=> /usr/bin/apxs MP_CCOPTS => -g -O2 -fdebug-prefix-map=/build/libapache2-mod-perl2-go9ZTa/libapache2-mod-perl2-2.0.10=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -fgnu89-inline MP_COMPAT_1X => 1 MP_GENERATE_XS => 1 MP_LIBNAME => mod_perl MP_TRACE => 0 MP_USE_DSO => 1 MP_USE_STATIC => 0 *** The httpd binary was not found *** (apr|apu)-config linking info -L/usr/lib/x86_64-linux-gnu -laprutil-1 -L/usr/lib/x86_64-linux-gnu -lapr-1 *** /usr/bin/perl -V Summary of my perl5 (revision 5 version 24 subversion 1) configuration: Platform: osname=linux, osvers=3.16.0, archname=x86_64-linux-gnu-thread-multi uname='linux localhost 3.16.0 #1 smp debian 3.16.0 x86_64 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dcc=x86_64-linux-gnu-gcc -Dcpp=x86_64-linux-gnu-cpp -Dld=x86_64-linux-gnu-gcc -Dccflags=-DDEBIAN -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/perl-N8J5tr/perl-5.24.1=. -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-z,relro -Dlddlflags=-shared
Bug#905603: libapache2-mod-perl2: crash when using EV::signal 'CHLD' and backtick operator at the same time
Package: libapache2-mod-perl2
Version: 2.0.10-2
Severity: important
Tags: patch upstream
-- Package-specific info:
-8<-- Start Bug Report 8<--
1. Problem Description:
To reproduce the problem, start from a vanilla Debian server (I used 9.5).
Install apache2, libapache2-mod-perl2, libev-perl
Create 2 perl files with following content:
handler1.pl:
#!/usr/bin/perl
my $pwd = `/bin/pwd`;
1;
handler2.pl:
#!/usr/bin/perl
use EV;
my $w = EV::signal 'CHLD', sub {
print "SIGCHLD received\n";
};
1;
Reference them in two PerlRequire directive, either in
/etc/apache2/apache2.conf or in a separate conf file in
/etc/apache2/conf-enabled:
# order is important
PerlRequire "/path/to/handler1.pl"
PerlRequire "/path/to/handler2.pl"
Apache will crash on startup.
===
I did an in-depth analysis of this bug. What happens is the following:
The backtick operator in handler1.pl internally calls waitpid, which
internally waits for the SIGCHLD signal.
EV::signal 'CHLD' installs a signal handler using sigaction (see
libev-perl-4.22: libev/ev.c::ev_signal_start). There is no provision to remove
this signal handler. The handler is located in EV.so, loaded by the XS loader.
apache2 goes through 3 stages on startup: pre-config, destroy-config,
create-config
In the pre-config stage, the 2 PerlRequire commands are executed and the
signal handler is installed at process level.
In the destroy-config stage, all modules are removed, including mod_perl,
which consists in removing all XS modules (see libapache2-mod-perl2-2.0.10:
src/modules/perl/modperl_interp.c::modperl_interp_destroy())
At this point the signal handler is still registered at process level but
points to invalid memory!
In the create-config stage, the config is recreated, handler1.pl is reloaded
and the backtick operator triggers a SIGCHLD, which causes a SIGEV because the
signal handler is not in memory (the EV.so library will only be loaded when the
second PerlRequire is executed).
I came to the conclusion that unloading the XS modules when mod_perl is
unloaded is very harsh; the perl interpretor is clearly not designed for that.
I tested a patch where the unloading of XS modules is simply skipped. It's
working and it also fixes another bug that I will report separately:
--- a/src/modules/perl/modperl_interp.c 2018-08-03 20:49:54.0 +0200
+++ b/src/modules/perl/modperl_interp.c 2018-08-03 20:49:54.0 +0200
@@ -122,11 +122,11 @@
MP_TRACE_i(MP_FUNC, "*error - still in use!*");
}
-handles = modperl_xs_dl_handles_get(aTHX);
+/*handles = modperl_xs_dl_handles_get(aTHX);*/
modperl_perl_destruct(interp->perl);
-modperl_xs_dl_handles_close(handles);
+/*modperl_xs_dl_handles_close(handles);*/
free(interp);
}
2. Used Components and their Configuration:
*** mod_perl version 2.10
*** using /usr/lib/x86_64-linux-gnu/perl5/5.24/Apache2/BuildConfig.pm
*** Makefile.PL options:
MP_APR_CONFIG => /usr/bin/apr-config
MP_APR_LIB => aprext
MP_APXS=> /usr/bin/apxs
MP_CCOPTS => -g -O2
-fdebug-prefix-map=/build/libapache2-mod-perl2-go9ZTa/libapache2-mod-perl2-2.0.10=.
-fstack-protector-strong -Wformat -Werror=format-security -Wall -fgnu89-inline
MP_COMPAT_1X => 1
MP_GENERATE_XS => 1
MP_LIBNAME => mod_perl
MP_TRACE => 0
MP_USE_DSO => 1
MP_USE_STATIC => 0
*** The httpd binary was not found
*** (apr|apu)-config linking info
-L/usr/lib/x86_64-linux-gnu -laprutil-1
-L/usr/lib/x86_64-linux-gnu -lapr-1
*** /usr/bin/perl -V
Summary of my perl5 (revision 5 version 24 subversion 1) configuration:
Platform:
osname=linux, osvers=3.16.0, archname=x86_64-linux-gnu-thread-multi
uname='linux localhost 3.16.0 #1 smp debian 3.16.0 x86_64 gnulinux '
config_args='-Dusethreads -Duselargefiles -Dcc=x86_64-linux-gnu-gcc
-Dcpp=x86_64-linux-gnu-cpp -Dld=x86_64-linux-gnu-gcc -Dccflags=-DDEBIAN
-Wdate-time -D_FORTIFY_SOURCE=2 -g -O2
-fdebug-prefix-map=/build/perl-N8J5tr/perl-5.24.1=. -fstack-protector-strong
-Wformat -Werror=format-security -Dldflags= -Wl,-z,relro -Dlddlflags=-shared
-Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.24 -Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.24
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.24 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.24.1
-Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.24.1
-Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
-Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3
-Dusesitecustomize -Duse64bitint -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm
