Bug#1102928: mpich: mpi-defaults autopkgtest regressions on 32-bit
Hi, On 24-04-2025 11:24, Paul Gevers wrote: On 24-04-2025 10:32, Alastair McKinstry wrote: My perference was to leave 4.3.0. in unstable, but given the valgrind issue, it looks like an upload 4.3.0_really4.2.1 is needed. Anyone think differently? I agree that a 4.3.0_really4.2.1 looks best at this moment. It would be great if an upload would happen soon. There are a lot of packages blocked behind mpich. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1102928: mpich: mpi-defaults autopkgtest regressions on 32-bit
Hi, On 24-04-2025 10:32, Alastair McKinstry wrote: My perference was to leave 4.3.0. in unstable, but given the valgrind issue, it looks like an upload 4.3.0_really4.2.1 is needed. Anyone think differently? I agree that a 4.3.0_really4.2.1 looks best at this moment. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1102928: mpich: mpi-defaults autopkgtest regressions on 32-bit
(presumably with some substitutions in the source code to get the appropriate multiarch tuple for the relevant architecture) would be enough to fix what we're now calling #1102928. I haven't verified that but it seems plausible. See also #1088919 and #1089694 which involve packaging changes that were made in 4.2.x in unstable, migrated to testing successfully, but then were lost when 4.3.x was re-uploaded from experimental into unstable. Those packaging changes seem like they might be relevant here. I'll check. I will upload 4.3.x to experimental with Adrians fix. (and debian/experimental branch) I understand your concern about the late stage of the release cycle, but I also notice that mpich 4.3.x was first uploaded to unstable 2 days before the toolchain freeze, which doesn't seem like an ideal time to be dropping functionality that previously existed and may have been relied on by dependent packages. I think that given circumstances 4.3.* is not ready for trixie. Do you plan to revert the upload of 4.3.x to unstable by uploading a 4.3.0+really4.2.1 version that matches what's in trixie, or is your intention that 4.3.x will stay in unstable, without migrating, until after the trixie release? The problem with having packages in unstable that are not ready for trixie is that whenever a dependent package is built on the buildds, it will be built against the dependency in unstable. For example, in this case, valgrind has a RC bug fix that *is* desired in trixie, but there is no way to avoid valgrind being compiled against mpich 4.3.x (at least on 32-bit architectures), so valgrind is subject to any compile-time regression that exists in 4.3.x - in particular the disappearance of mpi-c.pc. Depending on implementation details of the MPI ABI, it's possible that dependent packages like valgrind might also pick up a versioned dependency on libmpich12 (>= 4.3.0), which would prevent them from migrating before mpich does. (I don't know whether this is something that would genuinely happen for this particular package.) As a result of factors like those, the release team does not like packages being uploaded to unstable prematurely, and considers it to be a RC bug for a package to be in unstable for 30+ days without being able to migrate (#1103419). smcv (not a release team member, just a DD trying to help make the release happen) My perference was to leave 4.3.0. in unstable, but given the valgrind issue, it looks like an upload 4.3.0_really4.2.1 is needed. Anyone think differently? Alastair -- Alastair McKinstry, GPG: 82383CE9165B347C787081A2CBE6BB4E5D9AD3A5 e: alast...@mckinstry.ie, im: @alastair:mckinstry.ie @amckins...@mastodon.ie Commander Vimes didn’t like the phrase “The innocent have nothing to fear,” believing the innocent had everything to fear, mostly from the guilty but in the longer term even more from those who say things like “The innocent have nothing to fear.” - T. Pratchett, Snuff
Bug#1102928: mpich: mpi-defaults autopkgtest regressions on 32-bit
Control: clone 1102928 -2 Control: retitle 1102928 mpich: mpi-c.pc no longer available, causing mpi-defaults autopkgtest regression Control: retitle -2 mpich: mpi-fort.pc no longer available, causing mpi-defaults autopkgtest regression On Wed, 23 Apr 2025 at 10:58:30 +0100, Alastair McKinstry wrote: On 23/04/2025 10:47, Simon McVittie wrote: Fixing the C regression by reinstating the first --slave line seems considerably simpler, and might be enough to fix valgrind on 32-bit. Should we clone this bug into a C part which can certainly be fixed, and a Fortran part which might need further discussion? Yes, lets go with this. Cloning as requested. Please use #1102928 to represent the C side of this (which needs fixing relatively soon so that valgrind can migrate), and the new clone with a higher bug number to represent the Fortran side (which from what you've said might be wontfix). Adrian suggested that reinstating these arguments in the call to update-alternatives: --slave /usr/lib/i386-linux-gnu/pkgconfig/mpi-c.pc mpi-c.pc-i386-linux-gnu /usr/lib/i386-linux-gnu/pkgconfig/mpich.pc (presumably with some substitutions in the source code to get the appropriate multiarch tuple for the relevant architecture) would be enough to fix what we're now calling #1102928. I haven't verified that but it seems plausible. See also #1088919 and #1089694 which involve packaging changes that were made in 4.2.x in unstable, migrated to testing successfully, but then were lost when 4.3.x was re-uploaded from experimental into unstable. Those packaging changes seem like they might be relevant here. I understand your concern about the late stage of the release cycle, but I also notice that mpich 4.3.x was first uploaded to unstable 2 days before the toolchain freeze, which doesn't seem like an ideal time to be dropping functionality that previously existed and may have been relied on by dependent packages. I think that given circumstances 4.3.* is not ready for trixie. Do you plan to revert the upload of 4.3.x to unstable by uploading a 4.3.0+really4.2.1 version that matches what's in trixie, or is your intention that 4.3.x will stay in unstable, without migrating, until after the trixie release? The problem with having packages in unstable that are not ready for trixie is that whenever a dependent package is built on the buildds, it will be built against the dependency in unstable. For example, in this case, valgrind has a RC bug fix that *is* desired in trixie, but there is no way to avoid valgrind being compiled against mpich 4.3.x (at least on 32-bit architectures), so valgrind is subject to any compile-time regression that exists in 4.3.x - in particular the disappearance of mpi-c.pc. Depending on implementation details of the MPI ABI, it's possible that dependent packages like valgrind might also pick up a versioned dependency on libmpich12 (>= 4.3.0), which would prevent them from migrating before mpich does. (I don't know whether this is something that would genuinely happen for this particular package.) As a result of factors like those, the release team does not like packages being uploaded to unstable prematurely, and considers it to be a RC bug for a package to be in unstable for 30+ days without being able to migrate (#1103419). smcv (not a release team member, just a DD trying to help make the release happen)
Bug#1102928: mpich: mpi-defaults autopkgtest regressions on 32-bit
On 23/04/2025 10:47, Simon McVittie wrote:
On Sun, 13 Apr 2025 at 13:14:49 +0300, Adrian Bunk wrote:
214s autopkgtest [01:51:54]: test mpi-compile-run-cc-pkgconf-mpi-c:
[---
214s Package mpi-c was not found in the pkg-config search path.
...
This is related to two lines disappearing from the
libmpich-dev postinst:
--slave /usr/lib/i386-linux-gnu/pkgconfig/mpi-c.pc
mpi-c.pc-i386-linux-gnu /usr/lib/i386-linux-gnu/pkgconfig/mpich.pc \
--slave /usr/lib/i386-linux-gnu/pkgconfig/mpi-fort.pc
mpi-fort.pc-i386-linux-gnu /usr/lib/i386-linux-gnu/pkgconfig/mpich.pc
Re-adding the first line fixes mpi-compile-run-cc-pkgconf-mpi-c.
Fixing mpi-compile-run-f90-pkgconf-mpi* requires re-adding the second
line,
plus reverting the following change:
/usr/lib/i386-linux-gnu/mpich/lib/pkgconfig/mpich.pc
-Libs: -Wl,-z,relro -Wl,-z,now -L${libdir} -lmpichfort -lmpich
-lpthread -lhwloc
+Libs: -Wl,-z,relro -Wl,-z,now -L${libdir} -lmpich -Wl,-z,relro
-Wl,-z,now -lpthread -lhwloc
On Wed, 16 Apr 2025 at 12:07:36 +0100, Alastair McKinstry wrote:
I'm reluctant to make these changes for mpich as it recently
(upstream) dropped mpi-fort.pc support.
I believe the absence of mpi-c.pc might also be what is causing valgrind
to FTBFS on 32-bit architectures, preventing an unrelated RC bug fix from
migrating to testing (which I've reported as a separate bug, initially
against src:valgrind, but it might get reassigned to mpich).
Fixing the C regression by reinstating the first --slave line
seems considerably simpler, and might be enough to fix valgrind on
32-bit. Should we clone this bug into a C part which can certainly be
fixed, and a Fortran part which might need further discussion?
Yes, lets go with this.
(If the package still contains a libmpichfort that can be linked against,
then it doesn't actually seem particularly hard to retain backward
compatibility with mpi-fort.pc for trixie either, but perhaps there are
subtleties that I'm not aware of.)
I'd prefer we drop these tests in mpi-default for mpich in trixie
than add code to support in Debian, especially at this late stage.
Has this change been coordinated with mpi-defaults, and consumers of the
mpi-fort interface if any?
No, unfortunately.
I understand your concern about the late stage of the release cycle, but
I also notice that mpich 4.3.x was first uploaded to unstable 2 days
before the toolchain freeze, which doesn't seem like an ideal time to be
dropping functionality that previously existed and may have been
relied on
by dependent packages.
I think that given circumstances 4.3.* is not ready for trixie.
I haven't had word from upstream but am not sure the drop in
functionality is expected/permanent.
4.3.0-6 in experimental) fixes an important bug in mpich that should
replace -5 in sid, but think its probably best to leave 4.2* for trixie
smcv
Alastair
--
Alastair McKinstry,
GPG: 82383CE9165B347C787081A2CBE6BB4E5D9AD3A5
e: alast...@mckinstry.ie, im: @alastair:mckinstry.ie @amckins...@mastodon.ie
Commander Vimes didn’t like the phrase “The innocent have nothing to fear,”
believing the innocent had everything to fear, mostly from the guilty but in
the longer term
even more from those who say things like “The innocent have nothing to fear.”
- T. Pratchett, Snuff
Bug#1102928: mpich: mpi-defaults autopkgtest regressions on 32-bit
On Sun, 13 Apr 2025 at 13:14:49 +0300, Adrian Bunk wrote:
214s autopkgtest [01:51:54]: test mpi-compile-run-cc-pkgconf-mpi-c:
[---
214s Package mpi-c was not found in the pkg-config search path.
...
This is related to two lines disappearing from the
libmpich-dev postinst:
--slave /usr/lib/i386-linux-gnu/pkgconfig/mpi-c.pc mpi-c.pc-i386-linux-gnu
/usr/lib/i386-linux-gnu/pkgconfig/mpich.pc \
--slave /usr/lib/i386-linux-gnu/pkgconfig/mpi-fort.pc
mpi-fort.pc-i386-linux-gnu /usr/lib/i386-linux-gnu/pkgconfig/mpich.pc
Re-adding the first line fixes mpi-compile-run-cc-pkgconf-mpi-c.
Fixing mpi-compile-run-f90-pkgconf-mpi* requires re-adding the second line,
plus reverting the following change:
/usr/lib/i386-linux-gnu/mpich/lib/pkgconfig/mpich.pc
-Libs: -Wl,-z,relro -Wl,-z,now -L${libdir} -lmpichfort -lmpich -lpthread
-lhwloc
+Libs: -Wl,-z,relro -Wl,-z,now -L${libdir} -lmpich -Wl,-z,relro -Wl,-z,now
-lpthread -lhwloc
On Wed, 16 Apr 2025 at 12:07:36 +0100, Alastair McKinstry wrote:
I'm reluctant to make these changes for mpich as it recently
(upstream) dropped mpi-fort.pc support.
I believe the absence of mpi-c.pc might also be what is causing valgrind
to FTBFS on 32-bit architectures, preventing an unrelated RC bug fix from
migrating to testing (which I've reported as a separate bug, initially
against src:valgrind, but it might get reassigned to mpich).
Fixing the C regression by reinstating the first --slave line
seems considerably simpler, and might be enough to fix valgrind on
32-bit. Should we clone this bug into a C part which can certainly be
fixed, and a Fortran part which might need further discussion?
(If the package still contains a libmpichfort that can be linked against,
then it doesn't actually seem particularly hard to retain backward
compatibility with mpi-fort.pc for trixie either, but perhaps there are
subtleties that I'm not aware of.)
I'd prefer we drop these tests in mpi-default for mpich in trixie than
add code to support in Debian, especially at this late stage.
Has this change been coordinated with mpi-defaults, and consumers of the
mpi-fort interface if any?
I understand your concern about the late stage of the release cycle, but
I also notice that mpich 4.3.x was first uploaded to unstable 2 days
before the toolchain freeze, which doesn't seem like an ideal time to be
dropping functionality that previously existed and may have been relied on
by dependent packages.
smcv
Bug#1102928: mpich: mpi-defaults autopkgtest regressions on 32-bit
Source: buildd=mpich
Version: 4.3.0-5
Severity: serious
X-Debbugs-Cc: Alastair McKinstry
Control: affects src:mpi-defaults
https://tracker.debian.org/pkg/mpich
Issues preventing migration:
∙ ∙ autopkgtest for mpi-defaults/1.18: amd64: No tests, superficial or marked
flaky ♻ (reference ♻), arm64: No tests, superficial or marked flaky ♻
(reference ♻), armel: Regression or new test ♻ (reference ♻), armhf: Regression
or new test ♻ (reference ♻), i386: Regression or new test ♻ (reference ♻),
ppc64el: No tests, superficial or marked flaky ♻ (reference ♻), riscv64: No
tests, superficial or marked flaky ♻ (reference ♻), s390x: No tests,
superficial or marked flaky ♻ (reference ♻)
...
214s autopkgtest [01:51:54]: test mpi-compile-run-cc-pkgconf-mpi-c:
debian/tests/mpi-compile-run cc hello.c pkg-config mpi-c --libs --cflags
214s autopkgtest [01:51:54]: test mpi-compile-run-cc-pkgconf-mpi-c:
[---
214s Package mpi-c was not found in the pkg-config search path.
214s Perhaps you should add the directory containing `mpi-c.pc'
214s to the PKG_CONFIG_PATH environment variable
214s Package 'mpi-c', required by 'virtual:world', not found
214s autopkgtest [01:51:54]: test mpi-compile-run-cc-pkgconf-mpi-c:
---]
214s autopkgtest [01:51:54]: test mpi-compile-run-cc-pkgconf-mpi-c: - - - - -
- - - - - results - - - - - - - - - -
214s mpi-compile-run-cc-pkgconf-mpi-c FAIL non-zero exit status 1
...
261s + gfortran -o hello hello.f90 -I/usr/lib/arm-linux-gnueabihf/mpich/include
-L/usr/lib/arm-linux-gnueabihf/mpich/lib -lmpich -Wl,-z,relro -Wl,-z,now
-lpthread -lhwloc
261s /usr/bin/ld: /tmp/ccXCJF5d.o: in function `MAIN__':
261s hello.f90:(.text+0xc): undefined reference to `mpi_init_'
261s /usr/bin/ld: hello.f90:(.text+0x20): undefined reference to
`mpi_comm_size_'
261s /usr/bin/ld: hello.f90:(.text+0x34): undefined reference to
`mpi_comm_rank_'
261s /usr/bin/ld: hello.f90:(.text+0xaa): undefined reference to `mpi_finalize_'
261s collect2: error: ld returned 1 exit status
262s autopkgtest [01:52:42]: test mpi-compile-run-f90-pkgconf-mpi:
---]
262s autopkgtest [01:52:42]: test mpi-compile-run-f90-pkgconf-mpi: - - - - - -
- - - - results - - - - - - - - - -
262s mpi-compile-run-f90-pkgconf-mpi FAIL non-zero exit status 1
...
265s autopkgtest [01:52:45]: summary
265s mpi-compile-run-mpicc PASS (superficial)
265s mpi-compile-run-mpiCC PASS (superficial)
265s mpi-compile-run-mpif77 PASS (superficial)
265s mpi-compile-run-mpif90 PASS (superficial)
265s mpi-compile-run-cc-pkgconf-mpi PASS (superficial)
265s mpi-compile-run-cc-pkgconf-mpi-c FAIL non-zero exit status 1
265s mpi-compile-run-f90-pkgconf-mpi FAIL non-zero exit status 1
265s mpi-compile-run-f90-pkgconf-mpi-fort FAIL non-zero exit status 1
This is related to two lines disappearing from the
libmpich-dev postinst:
--slave /usr/lib/i386-linux-gnu/pkgconfig/mpi-c.pc mpi-c.pc-i386-linux-gnu
/usr/lib/i386-linux-gnu/pkgconfig/mpich.pc \
--slave /usr/lib/i386-linux-gnu/pkgconfig/mpi-fort.pc
mpi-fort.pc-i386-linux-gnu /usr/lib/i386-linux-gnu/pkgconfig/mpich.pc
Re-adding the first line fixes mpi-compile-run-cc-pkgconf-mpi-c.
Fixing mpi-compile-run-f90-pkgconf-mpi* requires re-adding the second line,
plus reverting the following change:
/usr/lib/i386-linux-gnu/mpich/lib/pkgconfig/mpich.pc
-Libs: -Wl,-z,relro -Wl,-z,now -L${libdir} -lmpichfort -lmpich -lpthread
-lhwloc
+Libs: -Wl,-z,relro -Wl,-z,now -L${libdir} -lmpich -Wl,-z,relro -Wl,-z,now
-lpthread -lhwloc
