On Mon, 2026-02-23 at 00:24 +0100, Tom Geelen via lists.openembedded.org wrote: > Based of a feature of AUH this is a plugin to run a testimage directly with > packages installed that you are working on via devtool. > > Inputs would be a: > - target image > - target packages > > The tool will take care to make sure it also installs the minimal necessary > dependencies to be able to run ptest on the target image. > Logs will be captured and stored in the devtool workspace for easy access. > > An oe-selftest is added to test the plugin. > > Signed-off-by: Tom Geelen <[email protected]> > --- > meta/lib/oeqa/selftest/cases/devtool.py | 61 +++++++++++ > scripts/lib/devtool/test_image.py | 130 ++++++++++++++++++++++++ > 2 files changed, 191 insertions(+) > create mode 100644 scripts/lib/devtool/test_image.py > > diff --git a/meta/lib/oeqa/selftest/cases/devtool.py > b/meta/lib/oeqa/selftest/cases/devtool.py > index d1209dd94e..c27934a3ca 100644 > --- a/meta/lib/oeqa/selftest/cases/devtool.py > +++ b/meta/lib/oeqa/selftest/cases/devtool.py > @@ -1932,6 +1932,67 @@ class DevtoolBuildImageTests(DevtoolBase): > if reqpkgs: > self.fail('The following packages were not present in the image > as expected: %s' % ', '.join(reqpkgs)) > > + > +class DevtoolTestImageTests(DevtoolBase): > + > + @OETestTag("runqemu") > + def test_devtool_test_image(self): > + """Test devtool test-image plugin.""" > + > + machine = get_bb_var('MACHINE') > + if not machine or not machine.startswith('qemu'): > + self.skipTest('This test only works with qemu machines') > + > + self.assertTrue(not os.path.exists(self.workspacedir), > + 'This test cannot be run with a workspace directory > under the build directory') > + > + image = 'oe-selftest-image' > + recipe = 'python3-atomicwrites'
I'm not sure that this is the best image recipe to use. Why not build
core-image-ptest-<packagename>?
> +
> + # Ensure selected test package is ptest-capable.
> + ptest_path = get_bb_var('PTEST_PATH', recipe)
> + self.assertTrue(ptest_path,
> + 'Selected package %s does not appear to inherit ptest' %
> recipe)
> +
> + self.track_for_cleanup(self.workspacedir)
> + # self.add_command_to_tearDown('bitbake -c clean %s' % image)
> + self.add_command_to_tearDown('bitbake-layers remove-layer
> */workspace')
> +
> + # Ensure we're starting from a clean state
> + bitbake('%s -c clean' % image)
> +
> + result = runCmd('devtool test-image %s -p %s' % (image, recipe),
> ignore_status=True)
> + if result.status != 0 and 'runqemu - ERROR - Unknown path arg' in
> result.output:
> + self.skipTest('runqemu in this environment does not accept
> testimage rootfs path args')
> + self.assertEqual(result.status, 0,
> + 'devtool test-image failed unexpectedly:\n%s' %
> result.output)
> +
> + # Check that requested package and its ptest package were installed
> + deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE')
> + self.assertTrue(deploy_dir_image, 'Unable to get DEPLOY_DIR_IMAGE')
> +
> + manifests = sorted(glob.glob(os.path.join(deploy_dir_image,
> '%s*.manifest' % image)))
> + self.assertTrue(manifests, 'Image manifest not found for %s in %s' %
> (image, deploy_dir_image))
> + manifest = manifests[-1]
> + self.assertExists(manifest, 'Image manifest not found: %s' %
> manifest)
> +
> + pkgs = set()
> + with open(manifest, 'r') as f:
> + for line in f:
> + splitval = line.split()
> + if splitval:
> + pkgs.add(splitval[0])
> +
> + self.assertIn(recipe, pkgs)
> + self.assertIn(recipe + '-ptest', pkgs)
> +
> + match = re.search(r'Logs are in (\S+)', result.output)
> + if match:
> + logdir = match.group(1)
> + else:
> + logdir = os.path.join(self.workspacedir, 'testimage-logs')
> + self.assertTrue(os.path.isdir(logdir), 'Expected logs directory not
> found: %s' % logdir)
> +
This test case took 10+ hours to run on the autobuilder [1]. Any ideas
why it's so slow?
[1]: https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3280
> +def test_image(args, config, basepath, workspace):
> + """Entry point for the devtool 'test-image' subcommand."""
> +
> + if not args.imagename:
> + raise DevtoolError('Image recipe to test must be specified')
> + if not args.package:
> + raise DevtoolError('Package(s) to install must be specified via
> -p/--package')
> +
> + package_names = [p.strip() for p in args.package.split(',') if p.strip()]
> + if not package_names:
> + raise DevtoolError('No valid package name(s) provided')
> +
> + install_pkgs = package_names
> +
> + logdir = os.path.join(config.workspace_path, 'testimage-logs')
> + try:
> + os.makedirs(logdir, exist_ok=True)
> + except Exception as exc:
> + raise DevtoolError(f'Failed to create test logs directory {logdir}:
> {exc}')
> +
> + pkg_append = ' '.join(sorted(set(install_pkgs)))
> + extra_append = [
> + f'TEST_LOG_DIR = "{logdir}"',
> + # Ensure runtime test framework is enabled even if image/distro
> omitted it
> + 'IMAGE_CLASSES += " testimage"',
> + # Ensure the testimage task has the correct IMAGE_FEATURES set in
> case the TEST_TARGET is qemu
> + 'IMAGE_FEATURES += "allow-empty-password empty-root-password
> allow-root-login"',
> + 'TEST_SUITES = " ping ssh ptest"',
> + 'TEST_RUNQEMUPARAMS += "slirp"',
This should not be set by default as different users may have different
needs. We need to rely on users setting the runqemuparams that the need
in local.conf/site.conf.
> + # Ensure image artifacts are built before do_testimage reads them.
> + 'do_testimage[depends] += " ${PN}:do_image_complete
> virtual/kernel:do_deploy"',
The do_testimage task should already have appropriate dependencies.
> + # Ensure rootfs link naming is runqemu-compatible for image names
> that
> + # otherwise end with '-image' (without a trailing '-').
> + 'IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"',
> + # Ensure a qemu-supported rootfs type is built/selected
> + 'IMAGE_FSTYPES:append = " ext4"',
> + 'QB_DEFAULT_FSTYPE = "ext4"',
Why do we need to force ext4 here?
> + # Enable ptests and include available -ptest packages for installed
> content
> + 'DISTRO_FEATURES:append = " ptest"',
> + 'IMAGE_FEATURES += " ptest-pkgs"',
> + 'IMAGE_INSTALL:append = " ptest-runner dropbear"',
We shouldn't assume dropbear here. If an ssh server is needed, using
"sshd" here should do the job and allow openssh or dropbear to be
chosen.
> + # Ensure requested packages (and -ptest where available) are
> installed
> + f'IMAGE_INSTALL:append = " {pkg_append}"',
> + ]
The extra_append argument to build_image_task is written to a bbappend
file for the image recipe, so it can't set global configuration like
DISTRO_FEATURES.
Best regards,
--
Paul Barker
signature.asc
Description: This is a digitally signed message part
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#232494): https://lists.openembedded.org/g/openembedded-core/message/232494 Mute This Topic: https://lists.openembedded.org/mt/117948896/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
