Still the systemd tests are very flaky.
I went through some re-runs and these last 7 tests all did test the very same
packages.
impish
amd64
boot-and-services (F 14% f 0% S 0% B 0% => P 85%/) ...F...
systemd-fsckd (F 0% f 0% S 100% B 0% => P 0%/)
This bug was fixed in the package qemu - 1:6.0+dfsg-1~ubuntu3
---
qemu (1:6.0+dfsg-1~ubuntu3) impish; urgency=medium
* d/p/u/lp-1935617-target-ppc-Fix-load-endianness-for-lxvwsx-lxvdsx.patch:
fix TCG emulation for ppc64 (LP: #1935617)
qemu (1:6.0+dfsg-1~ubuntu2) impish;
While some other flaky tests still hold it back the issue is fixed.
Here the PPC test of the new upload:
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/ppc64el/s/systemd/20210714_130923_091d6@/log.gz
--
You received this bug notification because you are a member of Ubuntu
Touch
Ok, now that we know the root cause we can mark systemd as invalid and the ppc
project as fixed.
The qemu task will be fixed once the upload migrates - but currently the test
queue is rather full.
** Changed in: systemd (Ubuntu)
Status: New => Invalid
** Changed in: ubuntu-power-systems
Ok, that fix works fine in a test.
I'm uploading this for a full test and hope that there are no further issues
with it that were hidden behind it so far.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
After some setup and preparation I was bisecting through v5.2.0..v6.0.0
to see which change exactly added this new behavior.
Summary:
- build reduced qemu
- build, configure and log-check errors will be skips
- run the extracted systemd test
- redirect serial to file
- check log if the
And that matches:
Offending commit:
bcb0b7b1a1c05707304f80ca6f523d557816f85c is the first new commit
commit bcb0b7b1a1c05707304f80ca6f523d557816f85c
Author: Giuseppe Musacchio
Date: Mon Nov 9 10:17:11 2020 +0100
ppc/translate: Rewrite gen_lxvdsx to use gvec primitives
Make the
** Tags removed: architecture-all targetmilestone-inin---
** Tags added: architecture-ppc64le targetmilestone-inin2110
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1935617
** Tags added: architecture-all bugnameltc-193617 severity-medium
targetmilestone-inin---
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1935617
Title:
systemd
** Also affects: ubuntu-power-systems
Importance: Undecided
Status: New
** Tags added: ppc64el reverse-proxy-bugzilla
** Changed in: ubuntu-power-systems
Assignee: (unassigned) => bugproxy (bugproxy)
** Changed in: ubuntu-power-systems
Importance: Undecided => High
--
You
I've collected the instructions that are around the failures and it is
suspicious for
lbzx / lvx / ld
All failures are around those:
Impish
ubuntu@i-systemdtest:/var/crash$ for i in */CoreDump; do p=$(dirname $i | tr -d
'.0123456789' | tr '_' '/' ); gdb $p $i -q -ex "x/-10i \$pc" -ex "x/3i
Hmm, I can blindly read through the qemu commit log and try rebuilds or even a
bisect at some point. But maybe this is already known. Since I'm soon on EOD I
wonder if we could mirror this to IBM (now that we know a bit more) if this is
a known issue to point us at an existing discussion or
We've had similar cases int he past where new releases glibc or compiler had
new instructions breaking the TCG mode.
So I spawned a focal and a bionic image - and then remodeled them to run in
emulation mode.
And indeed Bionic works without any crashes, so it must be something in the
guest
quotearg seems to be from gettext
https://sources.debian.org/src/gettext/0.21-4/gettext-tools/gnulib-lib/quotearg.c/?hl=1#L1
And is at least related in a few of the crashes (but not exclusively).
I wonder if the new TCG implements an instruction differently that now breaks.
--
You received
Backtraces
Two times (on rm, only slight variations)
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0798b9f361d0 in quotearg_buffer_restyled
(buffer=buffer@entry=0x798b9f60130 "'",
buffersize=buffersize@entry=256,
arg=arg@entry=0x800369e19807 ,
I installed a ppc64 impish cloud image (all fine) but then ran it in TCG
mode (to check if I can trigger the same).
That starts it like:
/usr/bin/qemu-system-ppc64le -name guest=i-systemdtest,debug-threads=on -S
-object
>From here I was switching to qemu 6.0 on the bare metal machine.
6.0-KVM - works
6.0-TCG - works
But running the testsuite itself fails (due to using only 512M ram).
I found that switching from testsuite-08 to testsuite-09 allowed me to get into
bad-mode in TCG (with enough memory, but into the
Since the bare metal case reports a memory issue on the crashes, let us
give it a bit more memory - with 2G it works finally.
Instead of the other tests in launchpad/canonistack this uses KVM mode
via "-machine accel=kvm -enable-kvm -cpu host". Switching that instead
to "-machine accel=tcg" it
I tried the same on a bare metal machine (now one is available for a
short time).
Got the new systemd source and test, ran it with qemu 5.2 and it is failing.
But not randomly, the same subset of tests fails on bare metal with 5.2.
But now they all fail on an OOM.
kvm: OS terminated: OS
The service has a StartPre that is like
ExecStartPre=rm -f /failed /testok
And that works to reproduce:
bash-5.1# rm -f /failed /testok
Segmentation fault (core dumped)
In fact this RM is very very sad:
-bash-5.1# rm /doesnotexist
Segmentation fault (core dumped)
-bash-5.1# touch /doesexist
As root:
$ mkdir /tmp/debugtest
$ export ARTIFACT_DIRECTORY=/tmp/debugtest
$ export TEST_SHOW_JOURNAL=warning
$ export TEST_REQUIRE_INSTALL_TESTS=0
$ export TEST_PREFER_NSPAWN=1
$ export NO_BUILD=1
$ export SELECTED_TESTS="TEST-08-*"
./test/run-integration-tests.sh
Reproduces the issue, it will
Isolated case
$ sudo env SELECTED_TESTS="TEST-08-*" debian/tests/upstream
It runs a VM Guest to do init stuff and that seems to break:
There are suspicious logs like systemd crashing, but those also exist
(to some extent) in the good case. I compared that:
$ grep -e "of user 0 dumped core" -e
While canonistack was breaking hard on me last week I managed to get an
instance running today.
Trying to reproduce the issue in there ...
Atfer a long while of upgrades, dependencies,
I was running with impish (as-is) vs impish+qemu6.0.
It was very reproucible working with 5.2 and always
> product engineering has access to PPC systems in maas?!? sigh...that sure
> would be nice
> for sustaining engineering to have access to also :(
That might sound more appealing than it is, the HW is often broken the
availability is worse than gold-nuggets and I think I never got a system
> Unfortunately the ppc maas seems down right now
product engineering has access to PPC systems in maas?!? sigh...that
sure would be nice for sustaining engineering to have access to also :(
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is
My Repro-tests indeed seem to indicate that really qemu 6.0 is the trigger for
this,
but as I said HW is rather unavailable at the moment. I was told that the MAAS
is serviced but should be back soon - so I'll have a look later when this one
is available again.
--
You received this bug
26 matches
Mail list logo
