[Bug 1831467] Re: test-umockdev tests flaky on armhf (and sometimes other archs)
marking as wontfix for xenial, as i won't get to it ** Changed in: umockdev (Ubuntu Xenial) Assignee: Dan Streetman (ddstreet) => (unassigned) ** Changed in: umockdev (Ubuntu Xenial) Status: In Progress => Won't Fix -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1831467] Re: test-umockdev tests flaky on armhf (and sometimes other archs)
This bug was fixed in the package umockdev - 0.11.1-1ubuntu0.1 --- umockdev (0.11.1-1ubuntu0.1) bionic; urgency=medium * d/p/lp1831467/0001-src-umockdev.vala-fix-ScriptRunner.op_write.patch, d/p/lp1831467/0002-tests-read-expected-number-of-bytes-maybe-over-multi.patch, d/p/lp1831467/0003-tests-test-umockdev-record.vala-add-slow_testbed_fac.patch, d/p/lp1831467/0004-tests-test-umockdev.c-add-slow_testbed_factor-variab.patch, d/p/lp1831467/0005-tests-test-umockdev-run.vala-add-slow_testbed_factor.patch, d/p/lp1831467/0006-remove_SocketServer_ctrl-r_assert.patch, d/rules, d/t/upstream: - Fix various timing-related tests Fix ScriptRunner fuzz checking function (LP: #1831467) -- Dan Streetman Thu, 06 Aug 2020 20:43:44 -0400 ** Changed in: umockdev (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1831467] Re: test-umockdev tests flaky on armhf (and sometimes other archs)
This bug was fixed in the package umockdev - 0.14.1-1ubuntu0.1 --- umockdev (0.14.1-1ubuntu0.1) focal; urgency=medium * d/p/lp1831467/0001-src-umockdev.vala-fix-ScriptRunner.op_write.patch, d/p/lp1831467/0002-tests-read-expected-number-of-bytes-maybe-over-multi.patch, d/p/lp1831467/0003-tests-test-umockdev-record.vala-add-slow_testbed_fac.patch, d/p/lp1831467/0004-tests-test-umockdev.c-add-slow_testbed_factor-variab.patch, d/p/lp1831467/0005-tests-test-umockdev-run.vala-add-slow_testbed_factor.patch, d/p/lp1831467/0006-remove_SocketServer_ctrl-r_assert.patch, d/rules, d/t/upstream: - Fix various timing-related tests Fix ScriptRunner fuzz checking function (LP: #1831467) -- Dan Streetman Thu, 06 Aug 2020 20:39:01 -0400 ** Changed in: umockdev (Ubuntu Focal) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1831467] Re: test-umockdev tests flaky on armhf (and sometimes other archs)
all tests pass for b and f (except the expected i386 failure on f, of course) ** Tags removed: verification-needed verification-needed-bionic verification-needed-focal ** Tags added: verification-done verification-done-bionic verification-done-focal -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1831467] Re: test-umockdev tests flaky on armhf (and sometimes other archs)
Hello Dan, or anyone else affected, Accepted umockdev into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/umockdev/0.14.1-1ubuntu0.1 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-focal. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: umockdev (Ubuntu Focal) Status: In Progress => Fix Committed ** Tags added: verification-needed verification-needed-focal ** Changed in: umockdev (Ubuntu Bionic) Status: In Progress => Fix Committed ** Tags added: verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1831467] Re: test-umockdev tests flaky on armhf (and sometimes other archs)
This bug was fixed in the package umockdev - 0.14.2-1 --- umockdev (0.14.2-1) unstable; urgency=medium * New upstream version 0.14.2: - preload: Fix incomplete reads scripts - tests: Fix incomplete reads for fuzzy script tests (Closes: #767908) - tests: Add $SLOW_TESTBED_FACTOR env var for slow architectures * Make autopkgtest robust for slow testbeds. Set the new `$SLOW_TESTBED_FACTOR` to avoid frequent failures on armhf and other slow/shared distro CI hardware. (LP: #1831467) * debian/rules: Add hack for code coverage related autoconf regression. -- Martin Pitt Wed, 29 Jul 2020 18:08:15 +0200 ** Changed in: umockdev (Ubuntu Groovy) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1831467] Re: test-umockdev tests flaky on armhf (and sometimes other archs)
https://salsa.debian.org/debian/umockdev/-/commit/87b476aee2 should hopefully help. I uploaded 0.14.2 to Debian unstable now, it should auto-sync into Groovy soon. Thanks Dan for tackling this! ** Changed in: umockdev (Ubuntu Groovy) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1831467] Re: test-umockdev tests flaky on armhf (and sometimes other archs)
** Description changed: [impact] - these tests fail intermittently, due to read failures when the test - expects to read data, but the umockdev 'replay' script device doesn't - provide the data to read. + these tests fail intermittently, due to various timing issues, as well + as an actual code bug in how data fuzz is calculated. - this appears to just be a timing issue, as the test expects the umockdev device to provide specific data replayed at specific times, and those times are very short (ms). Looking at the autopkgtest results: - http://autopkgtest.ubuntu.com/packages/umockdev - - it looks like the failures are mostly on armhf. + it looks like the failures are mostly on armhf, but do happen less often + on other archs. [test case] check the previous autopkgtest logs, e.g. https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/armhf/u/umockdev/20190601_015323_8f795@/log.gz [regression potential] - the tests on armhf have been flaky ever since trusty, and people just - retry and/or ignore them; any regression would likely result in (still) - flaky tests and/or false positive or negative test results on armhf. + any regression would likely result in incorrectly failing/passing autopkgtests, or in a incorrect pass or incorrect fail of the ScriptRunner + to verify the data's level of fuzz. [scope] as the tests are flaky on armhf all the way back to trusty, this is needed for all releases. [other info] - the armhf testbeds are unfortunately "different" from all the other archs: - 1) they don't run directly in a virtual instance, they run in an armhf lxd container inside an arm64 virtual instance. - 2) all other archs get are run inside a small instance, with limited memory and 1 cpu, but the armhf tests are run in what appears to be a large instance, and the armhf container inherits the 4 cpus that the arm64 instance has. + Fixed upstream in PR https://github.com/martinpitt/umockdev/pull/103 + + Most of the changes are test case fixes, but there is one fix to source + code to fix the ScriptRunning function that validates the level of fuzz + in data, so this is not *only* a testcase fix. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1831467] Re: test-umockdev tests flaky on armhf (and sometimes other archs)
** Also affects: umockdev (Ubuntu Groovy) Importance: Undecided Status: New ** Also affects: umockdev (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: umockdev (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: umockdev (Ubuntu Focal) Importance: Undecided Status: New ** Changed in: umockdev (Ubuntu Groovy) Assignee: (unassigned) => Dan Streetman (ddstreet) ** Changed in: umockdev (Ubuntu Focal) Assignee: (unassigned) => Dan Streetman (ddstreet) ** Changed in: umockdev (Ubuntu Bionic) Assignee: (unassigned) => Dan Streetman (ddstreet) ** Changed in: umockdev (Ubuntu Xenial) Assignee: (unassigned) => Dan Streetman (ddstreet) ** Changed in: umockdev (Ubuntu Groovy) Importance: Undecided => Low ** Changed in: umockdev (Ubuntu Focal) Importance: Undecided => Low ** Changed in: umockdev (Ubuntu Bionic) Importance: Undecided => Low ** Changed in: umockdev (Ubuntu Xenial) Importance: Undecided => Low ** Changed in: umockdev (Ubuntu Groovy) Status: New => In Progress ** Changed in: umockdev (Ubuntu Focal) Status: New => In Progress ** Changed in: umockdev (Ubuntu Bionic) Status: New => In Progress ** Changed in: umockdev (Ubuntu Xenial) Status: New => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1831467] Re: test-umockdev tests flaky on armhf (and sometimes other archs)
** Summary changed: - test-umockdev tests flaky on armhf + test-umockdev tests flaky on armhf (and sometimes other archs) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1831467 Title: test-umockdev tests flaky on armhf (and sometimes other archs) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/umockdev/+bug/1831467/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
