Package: libobject-remote-perl Version: 0.004001-2 Severity: serious Tags: ftbfs User: debian-p...@lists.debian.org Usertags: perl-5.36-transition
This package fails its test suite with Perl 5.36 on various architectures including i386, armel, armhf and ppc64el. This also makes the package fail to build from source on those architectures. t/basic.t ............. Attempt to reload XSLoader.pm aborted. Compilation failed in require at /usr/lib/arm-linux-gnueabi/perl-base/Cwd.pm line 79. Compilation failed in require at /usr/lib/arm-linux-gnueabi/perl-base/File/Spec/Unix.pm line 4. BEGIN failed--compilation aborted at /usr/lib/arm-linux-gnueabi/perl-base/File/Spec/Unix.pm line 4. Compilation failed in require at /usr/lib/arm-linux-gnueabi/perl-base/File/Spec.pm line 20. Compilation failed in require at /usr/share/perl5/Object/Remote/Connector/STDIO.pm line 3. BEGIN failed--compilation aborted at /usr/share/perl5/Object/Remote/Connector/STDIO.pm line 3. Compilation failed in require at /usr/share/perl5/Object/Remote/Node.pm line 4. BEGIN failed--compilation aborted at /usr/share/perl5/Object/Remote/Node.pm line 4. Compilation failed in require at (eval 1) line 29966. BEGIN failed--compilation aborted at (eval 1) line 29966. Channel closed without seeing Shere: eof at /usr/share/perl5/Object/Remote/Future.pm line 46. Dubious, test returned 255 (wstat 65280, 0xff00) No subtests run The reasons are fairly involved. Apologies for the wall of text. Executive summary: it's a long standing hidden Debian specific bug due to our Perl include path changes, somewhat accidentally mitigated on amd64 and some other architectures, and now triggered on the others due to an unrelated and innocent change in XSLoader. As I understand it, the module spawns another Perl process either locally or remotely and then sends it the code to run, including any necessary non-core modules, so that nothing needs to be installed from CPAN on the remote side. Clearly this cannot work for architecture specific code if the remote Perl process is running on a different architecture. So it looks like arch specific modules are purposefully filtered away from the code bundle, although I can't see this mentioned in the documentation. Now, the first problem here is filtering away the core modules. This is implemented by looking at loaded module filenames in %INC and matching them for the core include directory ($Config{privlib}). Unfortunately we have a Debian specific modification where /usr/bin/perl has the directory /usr/lib/<triplet>/perl-base early on the search path with a copy of part of the standard library [1], and the logic misses those and considers them as non-core modules. The above problem is mitigated on many architectures by the check for arch specific modules, which matches the filename against the known architecture specific directories in %Config (archlib, vendorarch, sitearch), $Config{archname} (with values like x86_64-linux-gnu-thread-multi or i686-linux-gnu-thread-multi-64int) and $Config{myarchname} (with values like x86_64-linux or i686-linux). The archname/myarchname part works (and is presumably intended) for the module directory structure created by ExtUtils::MakeMaker and friends for any extra include directories like ones managed with local::lib. As mentioned, the perl-base modules live in /usr/lib/<triplet>/perl-base, where <triplet> is the Debian "multiarch triplet" [2] with values like x86_64-linux-gnu or i386-linux-gnu. On architectures where $Config{myarchname} happens to match the multiarch triplet, the perl-base modules get filtered away as something of a side effect. This includes the most common x86_64 architecture known in Debian as "amd64", but not for instance the Debian "i386" or "armhf" architectures. All of the above has been the case since 2015 or so when the perl-base specific include path was introduced in Debian. It just hasn't caused any visible problems until now; possibly the extra core modules in the code bundle are not that architecture specific after all, and/or nobody just uses a mix of different architectures or Perl versions with Object::Remote (which sounds very probable.) Sending some extra core modules over has apparently just caused a bit of unintended overhead. The final straw that caused the regression with Perl 5.36 here is that XSLoader started to 'use strict'. To see why, we need to look at the code bundling (or "fatpacking") implementation a bit. This is lib/Object/Remote/FatNode.pm and it creates a string of code that is sent to the remote side where it gets eval'ed in. The code string is a series of here documents each containing the code of one module. These here documents are read into scalar Perl variables, turned into in-memory file handles [3], and eventually processed by 'require'. The in-memory handles need PerlIO::Scalar behind the scenes, which needs XSLoader.pm, which now needs strict.pm. So when strict.pm is "fatpacked" we now have a loop that Perl barfs over somehow. I'm a bit hazy on the further details, but a minimal if silly test case is unshift @INC, sub { if ($_[1] eq "strict.pm") { my $a = "1;"; open my $fh, '<', \$a }; return undef}; require strict; require XSLoader; print "ok\n"; which fails on 5.36 (regardless of the architecture) but not on 5.34. If I manually remove 'use strict' from XSLoader, it starts to pass on 5.36 as well. As for fixing this: I can't see a portable solution for filtering away the perl-base include path. It's a Debian specific modification and there is no Config entry to match against [4]. Even doing it as a Debian specific patch is a bit hard to do properly. I guess we could look up the multiarch triplet and construct the perl-base path manually, or do some mangling on $Config{archlib}, or just look for 'perl-base' in @INC and hope nobody uses that for local include directories. We could also try to do better in filtering away arch specific modules by matching for the multiarch triplet, with many of the same caveats as above. This might be made into something acceptable for upstream, though I suspect $Config{myarch} is only intended to match the EU::MM subdirectories. Lastly, a minimal change that keeps the unwanted bundling of perl-base modules but fixes this specific regression is to load strict.pm or preferrably all of PerlIO::Scalar before starting to evaluate the bundled code. This could be as easy as making a throwaway in-memory file handle before touching @INC. I've verified that it makes the test suite pass again on i386 without breaking it on amd64. I'm attaching such a change but I hesitate to call it a proper patch. Possibly we should still apply it for the time being, at least it shouldn't make things worse. (If somebody does, please make sure to fill in the bug number in the commit message.) I'm loath to pushing any of this upstream, though I suppose we should inform them that we have a mess here. It's really pretty much our (my) own fault. There's a reason for every Debian change that leads into this, but obviously it's something a systemic failure that we're hacking these perl-base specific things in without pushing for a generic solution upstream. If only perl-base wasn't Essential:yes we could get rid of most of the stuff. (Congratulations to everybody who managed to read this far!) [1] This has been the case since 5.22 Debian packaging in 2015 or so, and is done so that we can ship a full standard library for each major Perl version in libperl5.X and its dependencies for use with embedded Perl interpreters, but keep the perl-base package with /usr/bin/perl self contained and robust. [2] quoting dpkg-architecture(1) for DEB_HOST_MULTIARCH: The clarified GNU system type, used for filesystem paths. This triplet does not change even when the baseline ISA gets bumped, so that the resulting paths are stable over time. The only current difference with the GNU system type is that the CPU part for i386 based systems is always i386. Examples: i386-linux-gnu, x86_64-linux-gnu. Example paths: /lib/powerpc64le-linux-gnu/, /usr/lib/i386-kfreebsd-gnu/. [3] as in 'open my $fh, "<", \$scalar;' [4] it's not even in $Config{ccflags} because we ship a Config.pm from the shared build without it, see #798626 et al. -- Niko Tyni nt...@debian.org
>From 9b014783658e3023ed4ce75a5b679161784638ab Mon Sep 17 00:00:00 2001 From: Niko Tyni <nt...@debian.org> Date: Tue, 25 Oct 2022 18:27:05 +0100 Subject: [PATCH] Pre-load PerlIO::scalar before loading fatpacked modules This is just for robustness in case PerlIO::scalar or its dependencies somehow get included in the fatpacked bundle. See the Debian bug for a full story of how this could happen. Bug-Debian: https://bugs.debian.org/xxx --- lib/Object/Remote/FatNode.pm | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/Object/Remote/FatNode.pm b/lib/Object/Remote/FatNode.pm index cafc47e..fbc0c21 100644 --- a/lib/Object/Remote/FatNode.pm +++ b/lib/Object/Remote/FatNode.pm @@ -111,6 +111,10 @@ my $end = stripspace <<'END_END'; return; } + # pre-load PerlIO::scalar which we need for loading the fatpacked code + # just in case it or its dependencies got inadvertently bundled + open(my $fh, '<', \"1"); + unshift @INC, sub { load_from_hash(\%fatpacked, $_[1]) }; push @INC, sub { load_from_hash(\%fatpacked_extra, $_[1]) }; -- 2.37.2