Bug#1025868: lintian autopkgtest fails on all but amd64: x86_64-linux-gnu expected build-and-evaluate-test-packages/eval/checks/desktop/gnome/gir/gir/generic.t
Hi Axel, On 15-01-2023 23:07, Axel Beckert wrote: TL;DR: [...] You're awesome. And indeed, what a shame of your time. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1025868: lintian autopkgtest fails on all but amd64: x86_64-linux-gnu expected build-and-evaluate-test-packages/eval/checks/desktop/gnome/gir/gir/generic.t
Control: tag -1 + pending
Hi,
TL;DR:
The in-sbin-confusion-in-elf test in Lintian's test suite is broken
beyond repair: Lintian's arm64 result (which failed the test) is
actually the correct result, and the amd64 result (which passed the
test) is wrong due false negatives which lintian has no chance to
find. The cause are very likely different per-architecture compile
time string optimizations. So I will remove the test
in-sbin-confusion-in-elf from the test suite. The tag
bin-sbin-mismatch can't be tested on C-compiled binaries properly if
compile-time string optimizations are in use.
Long story including steps to get to that conclusion:
I've now digged into the second of these test suite failures on arm64
as I could neither reproduce it on armhf nor i386 (which were easier
available to me as I have Sid boxes with these architectures
permanently running).
And on arm64 I can indeed reproduce it. And it is much more weird than
I expected:
Paul Gevers wrote:
> # Hints do not match
> #
> # ---
> ../../autopkgtest_tmp/build-and-evaluate-test-packages/eval/checks/files/contents/bin-sbin-confusion-in-elf/hints.specified.calibrated
> # +++
> ../../autopkgtest_tmp/build-and-evaluate-test-packages/eval/checks/files/contents/bin-sbin-confusion-in-elf/hints.actual.parsed
> # +bin-sbin-confusion-in-elf (binary): bin-sbin-mismatch sbin/our-script ->
> usr/bin/our-script [usr/bin/calls-sbin]
> # +bin-sbin-confusion-in-elf (binary): bin-sbin-mismatch bin/our-script ->
> usr/bin/our-script [usr/bin/calls-sbin]
> #
> # Failed test 'Lintian passes for bin-sbin-confusion-in-elf'
> # at /usr/share/lintian/lib/Test/Lintian/Run.pm line 343.
> # Looks like you failed 1 test of 1.
> ../../autopkgtest_tmp/build-and-evaluate-test-packages/eval/checks/files/contents/bin-sbin-confusion-in-elf/generic.t
>
> Still need to figure out where the issue with bin-sbin-mismatch. But
> that tag is known (at least to me) for weird false positives.
The tag's implementation is not the cause. This seems not a direct bug
in Lintian.
So Lintian's test suite has a sample C program calls-sbin.c which the
test suite expects to trigger once. Here's the source code:
---8<---
#include
#include
/* may not work as expected on ELF due to ld's SHF_MERGE */
#define BIN_PATH "/bin/our-script"
#define SBIN_PATH "/sbin/our-script"
#define USR_BIN_PATH "/usr/bin/our-script"
#define USR_SBIN_PATH "/usr/sbin/our-script"
int
main(void)
{
printf("Calling %s\n", BIN_PATH);
printf("Calling %s\n", SBIN_PATH);
printf("Calling %s\n", USR_BIN_PATH);
printf("Calling %s\n", USR_SBIN_PATH);
return 0;
}
--->8---
Both files, our-script and calls-sbin are only installed into
/usr/bin/.
So how often should this tag trigger then? Once? Twice? Thrice?
The test suite thinks only once which I find confusing. I'd expected
at least twice. But the expected emitted tags by the test suite are:
bin-sbin-confusion-in-elf (binary): bin-sbin-mismatch usr/sbin/our-script ->
usr/bin/our-script [usr/bin/calls-sbin]
On arm64 these three tags are emitted:
bin-sbin-confusion-in-elf (binary): bin-sbin-mismatch usr/sbin/our-script ->
usr/bin/our-script [usr/bin/calls-sbin]
bin-sbin-confusion-in-elf (binary): bin-sbin-mismatch sbin/our-script ->
usr/bin/our-script [usr/bin/calls-sbin]
bin-sbin-confusion-in-elf (binary): bin-sbin-mismatch bin/our-script ->
usr/bin/our-script [usr/bin/calls-sbin]
Which is more what I would have expected on amd64 as well.
So far for the basic confusion.
Now to the more weird thing.
Not weird is IMHO that these four paths from the source are all in the
arm64 binary of calls-sbin:
[arm64] $ strings
./debian/test-out/packages/checks/files/contents/bin-sbin-confusion-in-elf/bin-sbin-confusion-in-elf-1.0/debian/bin-sbin-confusion-in-elf/usr/bin/calls-sbin
| fgrep bin
/bin/our-script
/sbin/our-script
/usr/bin/our-script
/usr/sbin/our-script
But in the amd64 binary, there are only two of them, with one being
the correct one:
[amd64] $ strings
./debian/test-out/packages/checks/files/contents/bin-sbin-confusion-in-elf/bin-sbin-confusion-in-elf-1.0/debian/bin-sbin-confusion-in-elf/usr/bin/calls-sbin
| fgrep bin
/usr/bin/our-script
/usr/sbin/our-script
This fits with the test suite only expecting this tag to be emitted.
But then again, the output of both architectures is the same:
[arm64] $
./debian/test-out/packages/checks/files/contents/bin-sbin-confusion-in-elf/bin-sbin-confusion-in-elf-1.0/debian/bin-sbin-confusion-in-elf/usr/bin/calls-sbin
Calling /bin/our-script
Calling /sbin/our-script
Calling /usr/bin/our-script
Calling /usr/sbin/our-script
[amd64] $
./debian/test-out/packages/checks/files/contents/bin-sbin-confusion-in-elf/bin-sbin-confusion-in-elf-1.0/debian/bin-sbin-confusion-in-elf/usr/bin/calls-sbin
Calling /bin/our-script
Calling /sbin/our-script
Calling /usr/bin/our-script
Calling /usr/sbin/our-script
So the more weird thing is: I
Bug#1025868: lintian autopkgtest fails on all but amd64: x86_64-linux-gnu expected build-and-evaluate-test-packages/eval/checks/desktop/gnome/gir/gir/generic.t
Hi, On 02-01-2023 21:10, Axel Beckert wrote: Another weird point seems that t/recipes/checks/desktop/gnome/gir/gir/eval/desc says: Testname: gir Check: desktop/gnome/gir Test-Architecture: amd64 So that clearly means it should only be run on amd64. So why is it run on arm64 at all? I suspect this is a lintian internal, but I guess you figured that out. Ah, ok. Yeah, kinda makes sense. I kinda expected that this version is usually the one the bug report was written against with any additional version hints being added as secondary version via Control statements. I cake these reports from a template and file them in with what I see on ci.d.n. P.S.: Any idea how we get Salsa CI autopkgtests on arm64? I understand the issue is on the salsa admin side where there are issues with shared runners or something. There is a host available that could run the tests, but the host can't be added for $reasons. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1025868: lintian autopkgtest fails on all but amd64: x86_64-linux-gnu expected build-and-evaluate-test-packages/eval/checks/desktop/gnome/gir/gir/generic.t
Hi Paul,
finally found time to tackle this.
Axel Beckert wrote:
> > Can you please investigate the situation
>
> Already done. Issue are mostly hardcoded x86_64 and amd64 strings in
> the test suite.
>
> Problem is IIRC that Lintian's testsuite currently doesn't support a
> templating system, at least not for many of these places. IIRC I fixed
> already some of them which were easy to fix.
Actually there seems a possibility to fix this like in these cases
t/recipes/checks/desktop/gnome/gir/gir/eval/post-test which has the
following sed command in it:
s, usr/lib/[^/${}]+/girepository-1.0$, usr/lib/MULTIARCH/girepository-1.0,
But it doesn't seem to be applied in the comparison, because if I put
"MULTIARCH" into the hints file, it fails on amd64 as well.
Another weird point seems that
t/recipes/checks/desktop/gnome/gir/gir/eval/desc says:
Testname: gir
Check: desktop/gnome/gir
Test-Architecture: amd64
So that clearly means it should only be run on amd64. So why is it run
on arm64 at all?
Hrm, a "git grep Test-Architecture" shows that most if not all other
such cases have "Test-Architectures" (plural with trailing "s") in
that place. So this is a typo which hasn't been caught yet.
So with my commit f415bc56c4b40d25410af21f2ea629e8eb0733b6 this part
of the test failures should be fixed. And with commit
695582b83f397d6bd5f47f99af77b2195ed1e604 we should also have an
internal check which finds such field name typos. (Needs to be
maintained, though, if fields get added or removed.)
Still need to figure out where the issue with bin-sbin-mismatch. But
that tag is known (at least to me) for weird false positives.
Paul Gevers wrote:
> On 10-12-2022 22:55, Axel Beckert wrote:
> > Ehm, that version no more in the archive anywhere. Did you maybe mean
> > 2.115.3 as currently in Testing and Unstable? (Feel free to remove the
> > moreinfo tag once this is clarified.)
>
> I meant, I see the issue *since* that version.
Ah, ok. Yeah, kinda makes sense. I kinda expected that this version is
usually the one the bug report was written against with any additional
version hints being added as secondary version via Control statements.
> But indeed, that's a bit weird if not commented on. I have added a
> `found` version now.
Thanks!
> > Will have to look into it again, but I fear in short term, it means to
> > either reduce the tests or only run a subset on non-amd64
> > architectures.
>
> If the test can't easily support other architectures (which is fine in my
> opinion) than please ensure it only runs on amd64. I advice to do that by
> adding a "Architecture: amd64" field to d/t/control.
Thanks for that hint. But there's already enough code in place for
that so that making it work should be merely fixing a few more lines.
The big question is which lines. ;-) But at least half of the issue is
fixed already.
P.S.: Any idea how we get Salsa CI autopkgtests on arm64?
Regards, Axel
--
,''`. | Axel Beckert , https://people.debian.org/~abe/
: :' : | Debian Developer, ftp.ch.debian.org Admin
`. `' | 4096R: 2517 B724 C5F6 CA99 5329 6E61 2FF9 CD59 6126 16B5
`-| 1024D: F067 EA27 26B9 C3FC 1486 202E C09E 1D89 9593 0EDE
Bug#1025868: lintian autopkgtest fails on all but amd64: x86_64-linux-gnu expected build-and-evaluate-test-packages/eval/checks/desktop/gnome/gir/gir/generic.t
Control: found -1 2.115.3 Control: tag -1 - moreinfo On 10-12-2022 22:55, Axel Beckert wrote: Ehm, that version no more in the archive anywhere. Did you maybe mean 2.115.3 as currently in Testing and Unstable? (Feel free to remove the moreinfo tag once this is clarified.) I meant, I see the issue *since* that version. But indeed, that's a bit weird if not commented on. I have added a `found` version now. Will have to look into it again, but I fear in short term, it means to either reduce the tests or only run a subset on non-amd64 architectures. If the test can't easily support other architectures (which is fine in my opinion) than please ensure it only runs on amd64. I advice to do that by adding a "Architecture: amd64" field to d/t/control. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1025868: lintian autopkgtest fails on all but amd64: x86_64-linux-gnu expected build-and-evaluate-test-packages/eval/checks/desktop/gnome/gir/gir/generic.t
Control: tag -1 + confirmed moreinfo Hi Paul, Paul Gevers wrote: > Source: lintian > Version: 2.111.0 Ehm, that version no more in the archive anywhere. Did you maybe mean 2.115.3 as currently in Testing and Unstable? (Feel free to remove the moreinfo tag once this is clarified.) > Your package has an autopkgtest, great. However, it fails currently > everywhere except on amd64. Correct. > Can you please investigate the situation Already done. Issue are mostly hardcoded x86_64 and amd64 strings in the test suite. Problem is IIRC that Lintian's testsuite currently doesn't support a templating system, at least not for many of these places. IIRC I fixed already some of them which were easy to fix. > and fix it? Will have to look into it again, but I fear in short term, it means to either reduce the tests or only run a subset on non-amd64 architectures. Anyway, thanks for the reminder. Regards, Axel -- ,''`. | Axel Beckert , https://people.debian.org/~abe/ : :' : | Debian Developer, ftp.ch.debian.org Admin `. `' | 4096R: 2517 B724 C5F6 CA99 5329 6E61 2FF9 CD59 6126 16B5 `-| 1024D: F067 EA27 26B9 C3FC 1486 202E C09E 1D89 9593 0EDE
Bug#1025868: lintian autopkgtest fails on all but amd64: x86_64-linux-gnu expected build-and-evaluate-test-packages/eval/checks/desktop/gnome/gir/gir/generic.t
Source: lintian Version: 2.111.0 Severity: serious User: debian...@lists.debian.org Usertags: regression Dear maintainer(s), Your package has an autopkgtest, great. However, it fails currently everywhere except on amd64. Can you please investigate the situation and fix it? I copied some of the output at the bottom of this report. The release team has announced [1] that failing autopkgtest on amd64 and arm64 are considered RC in testing as are regressions on all release architectures. More information about this bug and the reason for filing it can be found on https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation Paul [1] https://lists.debian.org/debian-devel-announce/2019/07/msg2.html https://ci.debian.net/data/autopkgtest/testing/arm64/l/lintian/28970519/log.gz # Hints do not match # # --- ../../autopkgtest_tmp/build-and-evaluate-test-packages/eval/checks/desktop/gnome/gir/gir/hints.specified.calibrated # +++ ../../autopkgtest_tmp/build-and-evaluate-test-packages/eval/checks/desktop/gnome/gir/gir/hints.actual.parsed # -gir1.2-good-42 (binary): typelib-not-in-multiarch-directory usr/lib/x86_64-linux-gnu/girepository-1.0 [usr/lib/girepository-1.0/GoodExtras-42.typelib] # -gir1.2-good-42 (binary): typelib-not-in-multiarch-directory usr/lib/x86_64-linux-gnu/girepository-1.0 [usr/lib/girepository-1.0/Good-42.typelib] # +gir1.2-good-42 (binary): typelib-not-in-multiarch-directory usr/lib/aarch64-linux-gnu/girepository-1.0 [usr/lib/girepository-1.0/GoodExtras-42.typelib] # +gir1.2-good-42 (binary): typelib-not-in-multiarch-directory usr/lib/aarch64-linux-gnu/girepository-1.0 [usr/lib/girepository-1.0/Good-42.typelib] # # Failed test 'Lintian passes for gir' # at /usr/share/lintian/lib/Test/Lintian/Run.pm line 343. # Looks like you failed 1 test of 1. ../../autopkgtest_tmp/build-and-evaluate-test-packages/eval/checks/desktop/gnome/gir/gir/generic.t ... and # Hints do not match # # --- ../../autopkgtest_tmp/build-and-evaluate-test-packages/eval/checks/files/contents/bin-sbin-confusion-in-elf/hints.specified.calibrated # +++ ../../autopkgtest_tmp/build-and-evaluate-test-packages/eval/checks/files/contents/bin-sbin-confusion-in-elf/hints.actual.parsed # +bin-sbin-confusion-in-elf (binary): bin-sbin-mismatch sbin/our-script -> usr/bin/our-script [usr/bin/calls-sbin] # +bin-sbin-confusion-in-elf (binary): bin-sbin-mismatch bin/our-script -> usr/bin/our-script [usr/bin/calls-sbin] # # Failed test 'Lintian passes for bin-sbin-confusion-in-elf' # at /usr/share/lintian/lib/Test/Lintian/Run.pm line 343. # Looks like you failed 1 test of 1. ../../autopkgtest_tmp/build-and-evaluate-test-packages/eval/checks/files/contents/bin-sbin-confusion-in-elf/generic.t OpenPGP_signature Description: OpenPGP digital signature
