On Mon, Mar 13, 2023 at 6:26 PM Alexander Kanavin <[email protected]> wrote:
> Tests in distrodata that contact upstream servers had to be disabled > because it's not possible to tell what is a real failure, and what is > a transient upstream issue (with expired certificates or other network > issues). To ensure upstream version checks don't regress, I run them > manually every now and then. > Thanks, can you confirm that git-submodule-test is currently failing for you (and probably was for long time) and that binutils is often reporting failure? On Mon, 13 Mar 2023 at 13:15, Martin Jansa <[email protected]> wrote: > > > > These changes are preparation for the actual changes from [YOCTO #12937], > > but without changing anything in current artifacts naming schema. > > > > Only change outside selftest was moving of INITRAMFS_IMAGE_NAME variable > > and adding IMAGE_MACHINE_SUFFIX variable to make it easier to change > both. > > > > I've run complete selftest with my other changes for [YOCTO #12937] which > > highlighted where the issues are and then again with only these changes > > to make sure I didn't break it for current mickledore. > > > > The most dangerous is probably: > > git-submodule-test: disable upstream version check > > because I cannot explain why distrodata.Distrodata.test_checkpkg isn't > failing on AB > > > > but maybe it's just because many (maybe all AB runs disable this > check?), e.g. > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/4948/steps/14/logs/stdio > > shows it skipped: > > ERROR: Command . ./oe-init-build-env; ${SCRIPTSDIR}/checkvnc; > OEQA_DEBUGGING_SAVED_OUTPUT=/srv/autobuilder/ > autobuilder.yocto.io/pub/repro-fail/ DISPLAY=:1 oe-selftest -a > --skip-tests distrodata.Distrodata.test_checkpkg > buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T > machine -T toolchain-user -T toolchain-system -j 15 failed with exit code > 1, see errors above. (1678628531.8: 4253.0) > > > > There were some tests which were skipped (even without --skip-tests) or > failed in my oe-selftest -a run: > > > > == SKIPPED == > > A) INFO - ... skipped 'Test needs meta-oe for nodejs' > > due to testing without meta-oe > > 2023-03-12 12:08:44,526 - oe-selftest - INFO - RESULTS - > devtool.DevtoolAddTests.test_devtool_add_npm: SKIPPED (2.22s) > > 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - > recipetool.RecipetoolCreateTests.test_recipetool_create_npm: SKIPPED (2.83s) > > > > B) INFO - ... skipped 'No "dash" found on test system.' > > no dash on gentoo > > 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - > oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: SKIPPED (0.00s) > > > > C) INFO - ... skipped 'Target does not use libssp' > > not testing on mingw32 with libssp: > > 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - > gcc.GccLibSspSelfTest.test_libssp: SKIPPED (3.47s) > > 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - > gcc.GccLibSspSelfTestSystemEmulated.test_libssp: SKIPPED (4.23s) > > > > D) INFO - ... skipped 'You must set up tap devices with > scripts/runqemu-gen-tapdevs before running this test' > > even when the tapdevs were correctly configured with > runqemu-gen-tapdevs before oe-selftest was started > > later resolved by "touch /etc/runqemu-nosudo" and it PASSED, > > > https://git.openembedded.org/openembedded-core/commit/?id=2258345e19efff7717fe19a5026ec55f1b6f90b6 > > looks suspicious and will try to improve it (to also ignore > > skipped taps like /tmp/qemu-tap-locks/tap0.skip which I'm using > > because tap0 is already used by openvpn running on my builder) > > 2023-03-12 12:08:44,526 - oe-selftest - INFO - RESULTS - > devtool.DevtoolExtractTests.test_devtool_deploy_target: SKIPPED (2.33s) > > > > == FAILED == > > 1) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - > distrodata.Distrodata.test_checkpkg: FAILED (146.68s) > > > > 2023-03-13 08:57:10,817 - oe-selftest - INFO - Traceback (most recent > call last): > > File "/OE/build/poky/meta/lib/oeqa/selftest/cases/distrodata.py", > line 40, in test_checkpkg > > self.assertTrue(len(regressed_failures) == 0 and > len(regressed_successes) == 0, msg) > > AssertionError: False is not true : > > The following packages failed upstream version checks. Please fix them > using UPSTREAM_CHECK_URI/UPSTREAM_CHECK_REGEX > > (when using tarballs) or UPSTREAM_CHECK_GITTAGREGEX (when using git). > If an upstream version check cannot be performed > > (for example, if upstream does not use git tags), you can set > UPSTREAM_VERSION_UNKNOWN to '1' in the recipe to acknowledge > > that the check cannot be performed. > > binutils > > > > used to show git-submodule-test as well (without the first patch here) > > I have no explanation why this isn't triggered on AB, when I've added > > more debug in this text, the binutils disappeared as well (so I was > > assuming it was just temporary network glitch, but today it returned > > again - with debug already removed). > > > > 2) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - > fetch.Fetch.test_git_mirrors: FAILED (24.34s) > > > > This is a limitation of current selftest as discussed with RP on IRC. > > > > easy to explain, but more difficult to fix, it sets > (PRE)MIRRORS:forcevariable = "" > > and expects to trigger fetch failure from dbus-wait, but > PREMIRRORS:prepend in my > > local.conf defeats it and dbus-wait is fetched successfully. > > > > I can switch from prepend to =+ in my local.conf but that will solve > it only in my environment. > > > > RP suggested to change selftest to ignore user's local.conf > > or pick only some specific key things from it, but current > > state is what it is and after removing PREMIRRORS:prepend > > this test PASSED as well. > > > > 3) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - > incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: > FAILED (2043.13s) > > > > 2023-03-13 09:01:25,976 - oe-selftest - INFO - ... FAIL > > 2023-03-13 09:01:25,977 - oe-selftest - INFO - Traceback (most recent > call last): > > File > "/OE/build/poky/meta/lib/oeqa/selftest/cases/incompatible_lic.py", line > 161, in test_core_image_full_cmdline_weston > > bitbake('-c testimage core-image-full-cmdline core-image-weston') > > File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 236, in > bitbake > > return runCmd(cmd, ignore_status, timeout, output_log=output_log, > **options) > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 214, in > runCmd > > raise AssertionError("Command '%s' returned non-zero exit status > %d:\n%s" % (command, result.status, exc_output)) > > AssertionError: Command 'bitbake -c testimage core-image-full-cmdline > core-image-weston' returned non-zero exit status 1: > > > > seems to run many tests succesfully with the test image: > > > > core-image-weston () - Ran 71 tests in 34.038s > > core-image-weston - OK - All required tests passed (successes=14, > skipped=57, failures=0, errors=0) > > NOTE: recipe core-image-weston-1.0-r0: task do_testimage: Succeeded > > NOTE: Tasks Summary: Attempted 1141 tasks of which 1139 didn't need to > be rerun and 1 failed. > > > > Summary: 1 task failed: > > > /OE/build/poky/meta/recipes-extended/images/core-image-full-cmdline.bb: > do_testimage > > Summary: There were 2 WARNING messages. > > Summary: There were 3 ERROR messages, returning a non-zero exit code. > > > > but there is also: > > runqemu - INFO - Found /tmp/qemu-tap-locks/tap0.skip, skipping tap0 > > runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap1.lock > failed: [Errno 11] Resource temporarily unavailable > > > > I need to debug it a bit more, probably outside selftest as well, will > try > > to run it when I don't need tap0 for openvpn. > > But it fails the same with current master without any of my changes as > well. > > > > 4) runqemu.RunqemuTests still fail with my other [YOCTO #12937] changes > > but work fine with this pull request, I have some WIP changes locally > > but the testimage+runqemu interaction is far from simple and will need > > more time to debug to cover all possible use-cases for runqemu > > > > runqemu.RunqemuTests.test_boot_machine_ext4.log:2023-03-13 09:03:12,008 > - oe-selftest - INFO - RESULTS - > runqemu.RunqemuTests.test_boot_machine_ext4: FAILED (63.37s) > > runqemu.RunqemuTests.test_boot_recipe_image.log:2023-03-13 08:31:06,400 > - oe-selftest - INFO - RESULTS - > runqemu.RunqemuTests.test_boot_recipe_image: ERROR (47.56s) > > runqemu.RunqemuTests.test_boot_recipe_image_vdi.log:2023-03-13 > 08:32:06,489 - oe-selftest - INFO - RESULTS - > runqemu.RunqemuTests.test_boot_recipe_image_vdi: ERROR (46.82s) > > runqemu.RunqemuTests.test_boot_recipe_image_vmdk.log:2023-03-13 > 08:33:07,020 - oe-selftest - INFO - RESULTS - > runqemu.RunqemuTests.test_boot_recipe_image_vmdk: ERROR (47.30s) > > > > The following changes since commit > 65ca2c4b7349a4f7dcfcc580d926a99c673deb60: > > > > dhcpcd: Fix install conflict when enable multilib. (2023-03-11 > 00:08:38 +0000) > > > > are available in the Git repository at: > > > > https://git.openembedded.org/openembedded-core-contrib jansa/pull > > > http://cgit.openembedded.org/openembedded-core-contrib/log/?h=jansa/pull > > > > Martin Jansa (14): > > git-submodule-test: disable upstream version check > > selftest: devtool: set BB_HASHSERVE_UPSTREAM when setting > > SSTATE_MIRROR > > selftest: wic: respect IMAGE_LINK_NAME > > selftest: wic: respect IMAGE_LINK_NAME also in > > test_rawcopy_plugin_qemu > > selftest: runqemu: respect IMAGE_LINK_NAME > > selftest: multiconfig-image-packager: try to respect IMAGE_LINK_NAME > > image-artifact-names.bbclass: add INITRAMFS_IMAGE_NAME from > > kernel.bbclass > > selftest: fitimage.py: respect INITRAMFS_IMAGE_NAME and > > KERNEL_FIT_LINK_NAME > > image-artifact-names: add IMAGE_MACHINE_SUFFIX variable > > selftest: gdbserver.py: respect IMAGE_LINK_NAME > > selftest: minidebuginfo.py respect IMAGE_LINK_NAME > > selftest: efibootpartition.py: fix QEMU_USE_KVM usage > > runqemu: get_first_file() rename cmd* to glob* > > selftest: imagefeatures.py: respect IMAGE_LINK_NAME for debugfs and > > manifest as well > > > > .../git-submodule-test/git-submodule-test.bb | 2 + > > .../multiconfig-image-packager_0.1.bb | 18 +++- > > meta-selftest/wic/test_rawcopy_plugin.wks.in | 2 +- > > .../image-artifact-names.bbclass | 16 +++- > > .../kernel-artifact-names.bbclass | 2 +- > > meta/classes-recipe/kernel.bbclass | 1 - > > meta/lib/oeqa/selftest/cases/devtool.py | 1 + > > .../oeqa/selftest/cases/efibootpartition.py | 4 +- > > meta/lib/oeqa/selftest/cases/fitimage.py | 34 ++++---- > > meta/lib/oeqa/selftest/cases/gdbserver.py | 9 +- > > meta/lib/oeqa/selftest/cases/imagefeatures.py | 82 +++++++++---------- > > meta/lib/oeqa/selftest/cases/minidebuginfo.py | 15 ++-- > > meta/lib/oeqa/selftest/cases/runqemu.py | 10 ++- > > meta/lib/oeqa/selftest/cases/wic.py | 44 +++++----- > > scripts/runqemu | 36 ++++---- > > 15 files changed, 155 insertions(+), 121 deletions(-) > > > > -- > > 2.39.2 > > > > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#178460): https://lists.openembedded.org/g/openembedded-core/message/178460 Mute This Topic: https://lists.openembedded.org/mt/97578950/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
