@evvers I think Ubuntu CI is at least mostly stable, for now, and
hopefully we can get the tests un-blacklisted soon, but since we're
doing all that work in github and debian bugs, I don't think we need to
keep this bug open, so I'll close it; let me know if you're prefer to
keep it open.
** Chang
opened https://github.com/systemd/systemd/issues/12932
so TEST-30 and TEST-34 need to go on the blacklist, unless we can figure
out how to get them less flaky soon.
** Bug watch added: github.com/systemd/systemd/issues #12932
https://github.com/systemd/systemd/issues/12932
--
You received th
> it looks like @laney is listed there, which is probably appropriate
(he has admin access to the test systems, while I don't)
My understanding is that @laney can help with the infrastructure where
the tests are run. What usually happens on Ubuntu CI for the most part
has nothing to do with the in
>From https://github.com/systemd/systemd/issues/12268#issuecomment-506953576
looks like TEST-30 is actually failing due to service start timeout (of 25
seconds); this test should be added to the blacklist until we can get it
reliably passing.
** Bug watch added: github.com/systemd/systemd/issues
Looking at the latest 20 i386 tests, I see failing tests for 12888,
12884, 12912, 12918, and 12919.
PR 12884 looks like it's adding a new test, TEST-35, so I'll assume that
test just needs more work.
PR 12888 failed once, but there are 3 later tests that all pass, so I
assume it's ok now.
PR 129
> That PR on GitHub hasn't been merged yet so I think it's too soon to
turn those tests off.
ack - I closed the MR to debian to blacklist them.
> Apparently on Ubuntu CI the tests are still run one by one
yep. I'll look at how Centos runs them to see if we can run them in
parallel on Ubuntu as
By the way, @ddstreet would it be OK to mention that you maintain Ubuntu
CI at https://www.freedesktop.org/wiki/Software/systemd/autopkgtest/?
Currently it's almost impossible to figure out who is responsible for
it. At some point I assumed it was @xnox but it doesn't seem the case so
I don't even
Regarding i386, I downloaded the log and took a look at what failed
there. Turns out all the tests except for TEST-34-DYNAMICUSERMIGRATE,
where the global timeout kicked in, passed. Apparently, to judge from
https://salsa.debian.org/systemd-
team/systemd/blob/master/debian/tests/upstream, on Ubuntu
> Opened MR to blacklist TEST-15 and TEST-22
That PR on GitHub hasn't been merged yet so I think it's too soon to
turn those tests off.
> ah, ok - so I did look into this, or at least some failures really
similar to this, a while back, for bug 1831468.
As far as I can tell, several tests failed
Opened MR to blacklist TEST-15 and TEST-22:
https://salsa.debian.org/systemd-team/systemd/merge_requests/35
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1829829
Title:
Ubuntu CI has been flaky for
> right now the upstream test timeouts on i386
ah, ok - so I did look into this, or at least some failures really
similar to this, a while back, for bug 1831468. As far as I could tell,
the problem in those cases was that the test was starting a service and
waiting for notification of its complet
Just to clarify, I think that it would be better to turn off the test
globally because we know it's flaky and unstable and we generally never
roll out globally flaky tests so as not to annoy contributors (I'm not
sure why Ubuntu CI should be different from any other CI system we use
upstream). And
Regarding the blacklist, we started to work on it in
https://github.com/systemd/systemd/issues/11195 (which was initially
about merging PRs where bugs were caught by Travis CI but ignored
because everybody was used to Ubuntu CI failing more often than not) and
ended up with a list of test I suggest
> Even if I'm totally wrong and someone is interested in getting it to
work on Ubuntu CI
I'll work on this; I would like it to work correctly.
> it should be possible to turn it off globally so as not to annoy
contributors with known issues
I agree about doing *something* for flaky tests - it ma
I agree ideally the test should be fixed but it's been flaky for a
couple of years and I didn't notice anyone who would be willing to try
to figure it out. I think it's time to admit nobody cares. Even if I'm
totally wrong and someone is interested in getting it to work on Ubuntu
CI, it should be p
> Would it also be possible to mark the "upstream" test "flaky" or turn
it off altogether
I'll have to defer to @xnox on that...personally I would prefer to leave
it on and try to fix whatever is causing the test(s) to be flaky, and/or
disable only individual upstream tests instead of marking all
In the meantime, I brought arm64 back almost as soon as
https://salsa.debian.org/systemd-team/systemd/merge_requests/34 was
merged. It looks promising except that the "upstream" test is flaky
there as well. It'd be great to turn it off on Ubuntu CI.
--
You received this bug notification because y
Good to know! Thank you!
Would it also be possible to mark the "upstream" test
(https://salsa.debian.org/systemd-
team/systemd/blob/master/debian/tests/control#L113) "flaky" or turn it
off altogether? We run it on CentOS and Arch anyway so it should be
safe. On Ubuntu CI it's particularly flaky an
> I think it would be better to somehow address
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824864.
> arm64 has been off since December 2018 due to that issue.
yep, the ubuntu bionic kernel will be patched to increase the arm64 log buffer
size:
https://lists.ubuntu.com/archives/kern
> looks like this has been fixed already upstream
Yes. I merged the PR (created by @yuwata) fixing that this morning.
> looks like someone stopped that; it's only running tests for 12888,
12897, and 12899 currently.
In
https://github.com/systemd/systemd/issues/12891#issuecomment-506093934 I
aske
> apparently something else was broken on bionic-i386 while it was off:
> https://github.com/systemd/systemd/issues/12891.
looks like this has been fixed already upstream
> In https://github.com/systemd/systemd/pull/12861#issuecomment-506025351 both
> bionic-amd64 and bionic-s390x failed due to
By the way, judging by http://autopkgtest.ubuntu.com/running#pkg-
systemd-upstream, it seems Ubuntu CI keeps running the tests for PR
12618, which was merged about a month ago.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bu
Anyway, as I said in
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1831296/comments/3,
I don't think Ubuntu CI in its current form is suitable for CI so I'll
just turn it off as soon as it starts failing again. I'm afraid I don't
have time for keeping an eye on it and reporting whatever co
I turned on bionic-amd64 and bionic-i386 yesterday. VMs no longer fail
to boot but apparently something else was broken on bionic-i386 while it
was off: https://github.com/systemd/systemd/issues/12891.
In https://github.com/systemd/systemd/pull/12861#issuecomment-506025351
both bionic-amd64 and bi
This seems to have 'fixed' itself (unless some fix was done without
mention in this bug), as the bionic systemd runs on amd64 have not had
this problem since june 3, and i386 since june 1.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu
** Changed in: systemd (Ubuntu)
Status: New => Confirmed
** Changed in: systemd (Ubuntu)
Importance: Undecided => High
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1829829
Title:
Ubuntu
To judge from
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic-upstream-systemd-ci-systemd-ci/bionic/i386/s/systemd-upstream/20190521_185445_b37c3@/log.gz,
VMs seems to also be throwing kernel panics:
```
[0.894903] BIOS EDD facility
@pitti would it be possible to temporarily skip the tests where VMs are
rebooted to reduce the blast radius so to speak? I really don't want to
turn Ubuntu CI off completely.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs
Indeed the downstream tests fail like this as well:
http://autopkgtest.ubuntu.com/packages/systemd/eoan/amd64
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1829829
Title:
Ubuntu CI has been flaky fo
29 matches
Mail list logo
